Service path selection in load balanced manner

ABSTRACT

Some embodiments provide novel methods for performing services for machines operating in one or more datacenters. For instance, for a group of related guest machines (e.g., a group of tenant machines), some embodiments define two different forwarding planes: (1) a guest forwarding plane and (2) a service forwarding plane. The guest forwarding plane connects to the machines in the group and performs L2 and/or L3 forwarding for these machines. The service forwarding plane (1) connects to the service nodes that perform services on data messages sent to and from these machines, and (2) forwards these data messages to the service nodes. In some embodiments, the guest machines do not connect directly with the service forwarding plane. For instance, in some embodiments, each forwarding plane connects to a machine or service node through a port that receives data messages from, or supplies data messages to, the machine or service node. In such embodiments, the service forwarding plane does not have a port that directly receives data messages from, or supplies data messages to, any guest machine. Instead, in some such embodiments, data associated with a guest machine is routed to a port proxy module executing on the same host computer, and this other module has a service plane port. This port proxy module in some embodiments indirectly can connect more than one guest machine on the same host to the service plane (i.e., can serve as the port proxy module for more than one guest machine on the same host).

BACKGROUND

Datacenters today use static, configuration intensive ways to distributedata messages between different application layers and to differentservice layers. A common approach today is to configure the virtualmachines to send packets to virtual IP (VIP) addresses, and thenconfigure the forwarding elements and load balancers in the datacenterwith forwarding rules that direct them to forward VIP addressed packetsto appropriate application and/or service layers. Another problem withexisting message distribution schemes is that today's load balancersoften are chokepoints for the distributed traffic. Accordingly, there isa need in the art for a new approach to seamlessly distribute datamessages in the datacenter between different application and/or servicelayers. Ideally, this new approach would allow the distribution schemeto be easily modified without reconfiguring the servers that transmitthe data messages.

BRIEF SUMMARY

Some embodiments provide novel methods for performing services formachines operating in one or more datacenters. For instance, for a groupof related guest machines (e.g., a group of tenant machines), someembodiments define two different forwarding planes: (1) a guestforwarding plane and (2) a service forwarding plane. The guestforwarding plane connects to the machines in the group and performs L2and/or L3 forwarding for these machines. The service forwarding plane(1) connects to the service nodes that perform services on data messagessent to and from these machines, and (2) forwards these data messages tothe service nodes.

In some embodiments, the guest machines do not connect directly with theservice forwarding plane. For instance, in some embodiments, eachforwarding plane connects to a machine or service node through a portthat receives data messages from, or supplies data messages to, themachine or service node. In such embodiments, the service forwardingplane does not have a port that directly receives data messages from, orsupplies data messages to, any guest machine. Instead, in some suchembodiments, data associated with a guest machine is routed to a portproxy module executing on the same host computer, and this port proxymodule has a service plane port. This port proxy module in someembodiments indirectly can connect more than one guest machine on thesame host to the service plane (i.e., can serve as the port proxy modulefor more than one guest machine on the same host).

In some embodiments, a guest machine is any machine that is not aservice machine or node. A guest machine can be a tenant's machine in amulti-tenant datacenter, but it does not have to be. A guest machine insome embodiments is a guest virtual machine or guest container. Aservice node in some embodiments is a service virtual machine, a servicecontainer or a service appliance. In some embodiments, a service nodeperforms a middlebox service operation, such as a firewall, an intrusiondetection system, an intrusion prevention system, a load balancer, anencryptor, a message monitor, a message collector, or any number ofother middlebox services. As such, a service as used in this document isany type of middlebox service operation in some embodiments.

The preceding Summary is intended to serve as a brief introduction tosome embodiments of the invention. It is not meant to be an introductionor overview of all inventive subject matter disclosed in this document.The Detailed Description that follows and the Drawings that are referredto in the Detailed Description will further describe the embodimentsdescribed in the Summary as well as other embodiments. Accordingly, tounderstand all the embodiments described by this document, a full reviewof the Summary, Detailed Description, the Drawings and the Claims isneeded. Moreover, the claimed subject matters are not to be limited bythe illustrative details in the Summary, Detailed Description and theDrawing.

BRIEF DESCRIPTION OF THE DRAWINGS

The novel features of the invention are set forth in the appendedclaims. However, for purposes of explanation, several embodiments of theinvention are set forth in the following figures.

FIG. 1 illustrates an example of segregated guest and service planesthat are implemented in some embodiments by two logical forwardingelements.

FIG. 2 illustrates a data message between two guest virtual machines(GVMs) being redirected along a service path to be processed by servicevirtual machines (SVMs) of some embodiments.

FIG. 3 conceptually illustrates a relationship between a service chainand a set of one or more service paths that implement the service chainin some embodiments.

FIG. 4 illustrates an example of a service chain and its associatedservice paths.

FIG. 5 illustrates examples of reverse service paths for the forwardservice paths illustrated in FIG. 4.

FIG. 6 illustrates an example of input/output (TO) chain components thatimplement a service plane in some embodiments.

FIG. 7 illustrates a process performed by a service index pre-processorand a service transport layer caller of some embodiments

FIG. 8 illustrates a data flow example corresponding to the processdescribed in FIG. 7.

FIG. 9 illustrates an operation of a port proxy of some embodiments forformatting a data message for forwarding by a first service node.

FIG. 10 conceptually illustrates a process of some embodiments forpassing a data message in a service path to a next hop.

FIG. 11 illustrates a process that the service proxy of FIG. 6 performsin some embodiments each time it receives a data message traversingalong an ingress path of a service node.

FIG. 12 conceptually illustrates three encapsulation headers of a datamessage of some embodiments.

FIG. 13 conceptually illustrates one exemplary process that an SVMperforms in some embodiments each time it receives a data message toprocess from a service proxy.

FIG. 14 illustrates a first mapping table of an SVM of some embodiments.

FIG. 15 illustrates an example of a data message in some embodimentsbeing forwarded from a first hop service node to a second hop servicenode.

FIG. 16 conceptually illustrates a process that a service proxy performsin some embodiments each time it receives a data message traversingalong an egress path of its service node.

FIG. 17 conceptually illustrates a process started by an encap processoron a next hop computer that receives an encapsulated data message thatneeds to be processed by an SVM executing on its computer.

FIG. 18 illustrates an example of a data message in some embodimentsbeing forwarded from a second hop service node to a third hop servicenode.

FIG. 19 illustrates an example of a data message in some embodimentsbeing forwarded from a third hop service node to a back to a first hopservice node.

FIG. 20 conceptually illustrates a process that a service indexpost-processor performs in some embodiments.

FIG. 21 illustrates a network service header of some embodiments.

FIG. 22 illustrates an example of metadata content that is stored in ametadata content header of some embodiments.

FIG. 23-24 illustrate an example of a service proxy forwarding to an SVMegress-side and ingress-side data messages of a GVM with encapsulatingGRE headers.

FIG. 25 illustrates a GRE header format that is used in some embodimentsto store service data for egress direction.

FIG. 26 illustrates a GRE header format that is used in some embodimentsto store service data for ingress direction.

FIG. 27 illustrate the use of two Geneve encapsulation headers, an outerGeneve header for carrying service transport layer data and an innerGeneve header for carrying service insertion layer metadata.

FIG. 28 illustrates the two Genece encapsulation headers of FIG. 27combined into a single Geneve encapsulation header.

FIG. 29 illustrates an object data model of some embodiments.

FIG. 30 conceptually illustrates several operations that networkmanagers and controllers perform in some embodiments to define rules forservice insertion, next service hop forwarding, and service processing.

FIG. 31 illustrates how service paths are dynamically modified in someembodiments.

FIG. 32 illustrates a process that some embodiments perform to define aservice plane and its associated service nodes for a tenant in amulti-tenant datacenter.

FIG. 33 conceptually illustrates an electronic system with which someembodiments of the invention are implemented.

DETAILED DESCRIPTION

In the following detailed description of the invention, numerousdetails, examples, and embodiments of the invention are set forth anddescribed. However, it will be clear and apparent to one skilled in theart that the invention is not limited to the embodiments set forth andthat the invention may be practiced without some of the specific detailsand examples discussed.

Some embodiments provide novel methods for performing services formachines operating in one or more datacenters. For instance, for a groupof related guest machines (e.g., a group of tenant machines), someembodiments define two different forwarding planes: (1) a guestforwarding plane and (2) a service forwarding plane. The guestforwarding plane connects to the machines in the group and performs L2and/or L3 forwarding for these machines. The service forwarding plane(1) connects to the service nodes that perform services on data messagessent to and from these machines, and (2) forwards these data messages tothe service nodes.

In some embodiments, the guest machines do not connect directly with theservice forwarding plane. For instance, in some embodiments, eachforwarding plane connects to a machine or service node through a portthat receives data messages from, or supplies data messages to, themachine or service node. In such embodiments, the service forwardingplane does not have a port that directly receives data messages from, orsupplies data messages to, any guest machine. Instead, in some suchembodiments, data associated with a guest machine is routed to a portproxy module executing on the same host computer, and this other modulehas a service plane port. This port proxy module in some embodimentsindirectly can connect more than one guest machine on the same host tothe service plane (i.e., can serve as the port proxy module for morethan one guest machine on the same host).

In some embodiments, a guest machine is any machine that is not aservice machine or node. A guest machine can be a tenant's machine in amulti-tenant datacenter, but it does not have to be. A guest machine insome embodiments is a guest virtual machine or guest container. Aservice node in some embodiments is a service virtual machine, a servicecontainer or a service appliance. In some embodiments, a service nodeperforms a middlebox service operation, such as a firewall, an intrusiondetection system, an intrusion prevention system, a load balancer, anencryptor, a message monitor, a message collector, or any number ofother middlebox services. As such, a service as used in this document isany type of middlebox service operation in some embodiments.

Also, as used in this document, data messages refer to a collection ofbits in a particular format sent across a network. One of ordinary skillin the art will recognize that the term data message is used in thisdocument to refer to various formatted collections of bits that are sentacross a network. The formatting of these bits can be specified bystandardized protocols or non-standardized protocols. Examples of datamessages following standardized protocols include Ethernet frames, IPpackets, TCP segments, UDP datagrams, etc. Also, as used in thisdocument, references to L2, L3, L4, and L7 layers (or layer 2, layer 3,layer 4, and layer 7) are references respectively to the second datalink layer, the third network layer, the fourth transport layer, and theseventh application layer of the OSI (Open System Interconnection) layermod

FIG. 1 illustrates an example of segregated guest and service planesthat are implemented in some embodiments by two logical forwardingelements (LFEs) 130 and 132. As shown, two guest machines 102 and 104and three service machines 106, 108 and 110 execute on three hostcomputers 112, 114 and 116 along with three software forwarding elements120, 122 and 124. In this example, the guest machines and servicemachines are guest virtual machines (GVMs) and service virtual machines(SVMs), but in other embodiments these machines can be other types ofmachines, such as containers.

Also, in this example, each logical forwarding element is a distributedforwarding element that is implemented by configuring multiple softwareforwarding elements (SFEs) on multiple host computers. To do this, eachSFE or a module associated with the SFE in some embodiments isconfigured to encapsulate the data messages of the LFE with an overlaynetwork header that contains a virtual network identifier (VNI)associated with the overlay network. As such, the LFEs are said to beoverlay network constructs that span multiple host computers in thediscussion below.

The LFEs also span in some embodiments configured hardware forwardingelements (e.g., top of rack switches). In some embodiments, each LFE isa logical switch that is implemented by configuring multiple softwareswitches (called virtual switches or vswitches) or related modules onmultiple host computers. In other embodiments, the LFEs can be othertypes of forwarding elements (e.g., logical routers), or any combinationof forwarding elements (e.g., logical switches and/or logical routers)that form logical networks or portions thereof. Many examples of LFEs,logical switches, logical routers and logical networks exist today,including those provided by VMware's NSX network and servicevirtualization platform.

As shown, the LFE 130 defines the guest forwarding plane that connectsthe GVMs 102 and 104 in order to forward data messages between theseGVMs. In some embodiments, this LFE is a logical switch that connects toa logical router, which connects the GVMs directly or through a logicalgateway to networks outside of the logical switch's logical network. TheLFE 130 is implemented in some embodiments by configuring softwareswitches 120 and 122 and/or their related modules (e.g., relatedport/VNIC filter modules) on the host computers 112 and 114 to implementa first distributed logical switch.

FIG. 1 and other figures discussed below show the source and destinationGVMs being on the same logical network and being connected to the sameLFE. One of ordinary skill will realize that the service operations ofsome embodiments do not require the source and destination machines tobe connected to the same LFE, or to even be in the same network or thesame datacenter. These service operations are performed on data messagesthat exit the source machine's network or enter a source machine'snetwork. The figures depict the source and destination machines asconnected to the same LFE to emphasize that the service plane 132 isimplemented by a separate logical network than the logical network thatforwards the data messages associated with the guest machines.

The LFE 132 defines the service forwarding plane that connects the SVMs106, 108 and 110 in order to forward data messages associated with theGVMs through service paths that include the SVMs. In some embodiments,the LFE 132 is also a logical switch that is implemented by configuringsoftware switches 120, 122 and 124 and/or their related modules on thehost computers 112, 114 and 116 to implement a second distributedlogical switch. Instead of configuring the same set of SFEs to implementboth the guest and service forwarding planes (i.e., the guest andservice LFEs), other embodiments configure one set of SFEs on a set ofhost computers to implement the guest forwarding plane and another setof SFEs on the set of host computers to implement the service forwardingplane. For instance, in some embodiments, each host computer executes aguest software switch and a service software switch, and these twoswitches and/or their related modules can be configured to implement aguest logical switch and a service logical switch.

In some embodiments, the software switches 120, 122 and 124 and/or theirrelated modules can be configured to implement multiple guest forwardingplanes (e.g., guest LFEs) and multiple service forwarding planes (e.g.,service LFEs) for multiple groups of machines. For instance, for amulti-tenant datacenter, some such embodiments define a guest LFE and aservice LFE for each tenant for which at least one chain of servicesneeds to be implemented. For each group of related machines (e.g., foreach tenant's machines), some embodiments define two virtual networkidentifiers (VNIs) to configure a shared set of software forwardingelements (e.g., software switches) to implement the two differentforwarding planes, i.e., the guest forwarding plane and the serviceforwarding plane. These two VNIs are referred to below as the guest VNI(GVNI) and the service VNI (SVNI). In FIG. 1, the guest LFE ports 150and 152 are associated with the GVNI, while the service LFE ports 154,156, and 158 are associated with the SVNI, as shown.

In some embodiments, the service plane 132 is also implemented byinserting modules in input/output (TO) chains of a GVM's egress andingress datapaths to and from an SFE 120 or 122. In this implementation,the service plane 132 can identify a data message sent from the GVM orreceived for the GVM, forward the data message to a set of SVMs toperform a chain of services on the data message, and then to return thedata message back to the GVM's datapath so that the data message can beproceed along its datapath to the software switch or to the GVM (i.e.,so that the data message can be processed based on the destinationnetwork addresses specified by the source GVM). Such a GVM is referredto below as the source GVM as the data message being processed by theservice nodes is a data message identified on the GVM's egress oringress path. In some embodiments, a GVM's egress/ingress IO chain isimplemented as a set of hooks (function calls) in the GVM's VNIC(virtual network interface card) 180 or the SFE port associated with theGVM's VNIC (e.g., the SFE port communicating with the GVM's VNIC).

Before providing an example of the IO chain components of someembodiments that implement the service plane, FIG. 2 illustrates anexample of a data message 202 from the GVM 102 to GVM 104 beingredirected along the service plane 132 so that the data message can beprocessed by SVMs 108 and 110 that perform a chain of two serviceoperations. As shown, the service LFE 132 first forwards the datamessage to SVM 108, and then forwards the data message to SVM 110,before returning the data message back to the egress path of GVM 102 sothat the data message can be processed based on the destination networkaddresses specified by the source GVM 102.

The service LFE in some embodiments forwards the data message betweenhosts 112, 114 and 116 by using an overlay encapsulation header thatstores the SVNI for the service LFE. Also, when the service LFE is aservice logical switch, the service forwarding plane in some embodimentsuses the MAC addresses associated with the SVMs (e.g., MAC addresses ofSVM VNICs) to forward the data message between ports of the servicelogical switch. In some embodiments, the MAC forwarding also usesservice plane MAC address associated with the source GVM, even thoughthis GVM does not directly connect to the service plane but insteadconnects to the service plane through a port proxy, as further describedbelow.

Once the data message 202 returns to the egress path of the GVM 102, theguest LFE 130 forwards the data message to its destination (e.g., asspecified by the destination network address in the data message'sheader), which is GVM 104. The guest LFE 130 in some embodimentsforwards the data message between hosts 112 and 114 by using an overlayencapsulation header that stores the GVNI for the guest LFE. Also, whenthe guest LFE is a logical switch, the guest forwarding plane in someembodiments uses the guest plane MAC addresses associated with the GVMs102 and 104 to forward the data message (e.g., by using the guest planeMAC address of GVM 104 to forward the data message to the guestforwarding port 152 associated with this GVM). While the service planeof FIG. 2 captures a data message passing through a GVM's egress path,the service plane in some embodiments can also capture a data message asit is passing through a GVM's ingress path before it reaches the GVM'sVNIC.

In some embodiments, a chain of service operations is referred to as aservice chain. A service chain in some embodiments can be implementedwith one or more sets of service nodes (e.g., service machines orappliances), with each set of service nodes defining a service path.Hence, in some embodiments, a service chain can be implemented by eachof one or more service paths. Each service path in some embodimentsincludes one or more service nodes for performing the set of one or moreservices of the service chain and a particular order through thesenodes.

FIG. 3 presents an object diagram that illustrates the relationshipbetween a service chain 302 and a set of one or more service paths 304that implement the service chain. Each service chain has a service chain(SC) identifier 306, while each service path has a service pathidentifier (SPI) 308. Each service path is associated with a set of mservice nodes, which, as shown, are identified in terms of serviceinstance endpoints 310. Service instance endpoints in some embodimentsare logical locations in the network where traffic can go or come from aservice node connected to the service plane. In some embodiments, aservice instance endpoint is one LFE port (e.g., an SFE port) associatedwith a service node (e.g., a VNIC of an SVM). In these or otherembodiments, a service instance endpoint can be associated with two LFEports used for a service node as further described below for embodimentsthat use GRE encapsulation. Also, the service endpoints in someembodiments are addressable through MAC addresses associated with theLFE ports or with the SVM VNICs associated with (e.g., communicatingwith these LFE ports).

In some embodiments, each service chain 302 is defined by references toone or more service profiles 312, with each service profile associatedwith a service operation in the chain. As described below, a servicenode in some embodiments (1) receives, from a service manager, a mappingof a service chain identifier to a service profile that it has toimplement, and (2) receives, with a data message, a service chainidentifier that it maps to the service profile to determine the serviceoperation that it has to perform. In some embodiments, the receivedmapping is not only based on the service chain identifier (SCI) but isalso based on a service index value (that specifies the location of theservice node in a service path) and a direction through a service chain(that specifies an order for performing the sequence of servicesspecified by the service chain). The service profile in some embodimentsdescribes the service operation that the service node has to perform. Insome embodiments, a service profile can identify a set of rules for aservice node to examine.

Also, in some embodiments, service insertion rules 314 are defined byreference to service chain identifies 306 for service insertion modulesassociated with GVMs. Such service insertion modules use these serviceinsertion rules 314 to identify service chains to use to process datamessages associated with a source GVM. As mentioned above, the datamessages are referred to below as being from a source GVM as the datamessages that are processed by the service chains are identified on theegress paths from or ingress paths to the GVMs.

As further described below, the service insertion (SI) rules associateflow identifiers with service chain identifiers. In other words, someembodiments try to match a data message's flow attributes to the flowidentifiers (referred to below as rule identifiers of the SI rules) ofthe service insertion rules, in order to identify a matching serviceinsertion rule (i.e., a rule with a set of flow identifiers that matchesthe data message's flow attributes) and to assign this matching rule'sspecified service chain as the service chain of the data message. Aspecific flow identifier (e.g., one defined by reference to a five-tupleidentifier) could identify one specific data message flow, while a moregeneral flow identifier (e.g., one defined by reference to less than thefive tuples) can identify a set of several different data message flowsthat match the more general flow identifier. As such, a matching datamessage flow is any set of data messages that have a common set ofattributes that matches a rule identifier of a service insertion rule.

As further described below, other embodiments use contextual attributesassociated with a data message flow to associate the data message with aservice insertion rule. Numerous techniques for capturing and usingcontextual attributes for performing forwarding and service operationsare described in U.S. patent application Ser. No. 15/650,251, which areincorporated herein. Any of these techniques can be used in conjunctionwith the embodiments described herein.

Next hop forwarding rules 316 in some embodiments are defined byreference to the SPI values 308 and service instance endpoints 310.Specifically, in some embodiments, a service path is selected for aservice chain that has been identified for a data message. At each hop,these embodiments use the forwarding rules 314 to identify the nextservice instance endpoint based on the SPI value for this service pathalong with a current service index (SI) value, which identifies thelocation of the hop in the service path. In other words, each forwardingrule in some embodiments has a set of matching criteria defined in termsof the SPI/SI values, and specifies a network address of the next hopservice instance endpoint that is associated with these SPI/SI values.To optimize the next hop lookup for the first hop, some embodimentsprovide to the source GVM's service insertion module the next hopnetwork address with the SPI, as part of a service path selectionprocess.

FIG. 4 illustrates an example of a service chain and its associatedservice path. As shown, each service chain 405 in some embodiments isdefined as a sequential list of service profiles 410, with each profilein this example related to a different middlebox service (such asfirewall, load balancer, intrusion detector, data message monitor,etc.). Also, in this example, each of the M profiles can be implementedby one SVM in a cluster m of VMs. As shown, different clusters fordifferent profiles can have different numbers of SVMs. Also, in someembodiments, one service profile is implemented by one service node(i.e., a cluster of several service nodes is not required to implement aservice profile).

Since multiple SVMs in a cluster can provide a particular service, someembodiments define for a given service chain, multiple service pathsthrough multiple different combinations of SVMs, with one SVM of eachcluster being used in each combination. In the example of FIG. 4, thereare N service paths associated with the service chain 405, traversed bydata messages originating at a GVM 402 on their way to a GVM 404. Eachservice path is identified by a different set of dashed lines in thisfigure.

Specifically, the first service path passes through first SVM 1,1 of thefirst service profile's cluster to implement the first service of theforward service chain 405, the first SVM 2,1 of the second serviceprofile's cluster to implement the second service of the forward servicechain 405, and third SVM M,3 of the Mth service profile's cluster toimplement the Mth service of the forward service chain 405. The secondservice path passes through second SVM 1,2 of the first serviceprofile's cluster to implement the first service of the forward servicechain 405, the first SVM 2,1 of the second service profile's cluster toimplement the second service of the forward service chain 405, and firstSVM M,1 of the Mth service profile's cluster to implement the Mthservice of the forward service chain 405.

The third service path passes through third SVM 1,3 of the first serviceprofile's cluster to implement the first service of the forward servicechain 405, the second SVM 2,2 of the second service profile's cluster toimplement the second service of the forward service chain 405, andsecond SVM M,2 of the Mth service profile's cluster to implement the Mthservice of the forward service chain 405. The Nth service path passesthrough third SVM 1,3 of the first service profile's cluster toimplement the first service of the forward service chain 405, the secondSVM 2,2 of the second service profile's cluster to implement the secondservice of the forward service chain 405, and fourth SVM M,4 of the Mthservice profile's cluster to implement the Mth service of the forwardservice chain 405. As the example illustrates, different service pathsmay use the same SVM for a given service operation. However, regardlessof the service path that a given data message traverses, the same set ofservice operations is performed in the same sequence, for paths that areassociated with the same service chain and the same service direction.

In some embodiments, a service chain has to be performed in a forwarddirection for data messages from a first GVM to a second GVM, and thenin the reverse direction for data messages from the second GVM to thefirst GVM. In some such embodiments, the service plane selects both theservice path for the forward direction and the service path for thereverse direction when it processes the first data message in the flowfrom the first GVM to the second GVM. Also, in some of theseembodiments, the forward and reverse service paths are implemented bythe same sets of service nodes but in the reverse order.

FIG. 5 illustrates examples of reverse service paths for the forwardservice paths illustrated in FIG. 4. While the forward service paths arefor performing M services on data messages from GVM 402 to GVM 404, thereverse service paths are for performing M services on data messagesfrom GVM 404 to GVM 402. Also, the order of these services is reversedwith the service paths in FIG. 5 performing service profiles M to 1,while the service paths in FIG. 4 perform service profile 1 to M.

Also, in the examples of FIGS. 4 and 5, each reverse service path hasone corresponding forward service path that is implemented by the sameexact set of SVMs but in the reverse order, as indicated by the servicepath legends and the similar dashed lines in these figures. For example,the forward, second service path passes through SVM 1,2 for the firstservice associated with the first profile, SVM 2,1 for the secondservice associated with the second profile, and SVM M,1 for the Mthservice associated with the Mth service profile, while the associatedreverse, second service path passes through SVM M,1 for the firstservice associated with the Mth service profile, SVM 2,1 for the secondservice associated with the second profile, and SVM 1,2 for the secondservice associated with the first profile.

In some embodiments, the same service nodes are used for the forward andreverse paths because at least one of the service nodes (e.g., afirewall SVM) that implements one of the service profiles needs to seethe data traffic in both directions between two data endpoints (e.g.,two GVMS). In other embodiments, the same service nodes do not need tobe used for both directions of data message flows between two dataendpoints so long as the same set of service operations are performed inopposite orders.

FIG. 6 illustrates an example of the IO chain components that implementthe service plane in some embodiments. As shown, the service plane 132is implemented by software switches 120, 122, and 124 executing on thehost computers and two sets of modules 610, 612, 614, 620, 624, 626, and628 on these computers. The implemented service plane in this example aswell some of the other examples illustrated in some of the subsequentfigures is an overlay logical L2 service plane. One of ordinary skillwill realize that other embodiments are implemented by other types ofservice planes, such as overlay L3 service planes, or overlay networkswith multiple L2 logical switches and one or more logical L3 routers.

In FIG. 6, the software switches 120, 122, and 124 and modules 610, 612,614, 620, 624, 626, and 628 implement two different layers of theservice plane, which are the service insertion layer 602 and the servicetransport layer 604. The service insertion layer 602 (1) identifies theservice chain for a data message, (2) selects the service path to use toperform the service operations of the service chain, (3) identifies thenext-hop service nodes at each hop in the selected service path(including the identification of the source host computer to which thedata message should be returned upon the completion of the servicechain), and (4) for the service path, specifies the service metadata(SMD) header attributes for the data message. The SMD attributes in someembodiments include the network service header (NSH) attributes per RFC(Request for Comments) 8300 of IETF (Internet Engineering Task Force).

The service transport layer 604, on the other hand, formulates theservice overlay encapsulation header and encapsulates the data messagewith this header so that it can pass between service hops. In someembodiments, the service transport layer 604 modifies the SMD header toproduce the service overlay encapsulation header. For instance, in someof these embodiments, the overlay encapsulation header is a Geneveheader with the SMD attributes stored in a TLV (type, length, value)section of the Geneve header. In other embodiments, the servicetransport layer 604 adds the service overlay encapsulation header to anSMD header that is first used to encapsulate the data message. Also,when traversing between two hops (e.g., between two service nodes)executing on the same host computer, the service transport layer inseveral embodiments described below does not encapsulate the datamessage with an overlay encapsulation header in some embodiments. Inother embodiments, even when traversing between two hops on the samehost computer, the service transport layer encapsulates the data messagewith an overlay encapsulation header.

In some embodiments, the service insertion (SI) layer 602 includes an SIpre-processor 610 and an SI post-processor 612, in each the two 10chains 650 and 652 (i.e., the egress IO chain 650 and the ingress IOchain 652) of a GVM for which one or more service chains are defined.The SI layer 602 also includes a service proxy 614 for each service nodeconnected to the service plane (e.g., for each SVM with a VNIC pairedwith a service plane LFE port). The service transport (ST) layer 604includes one STL port proxy 620 on each host computer that has one ormore possible source GVMs for which one or more service chains aredefined. The ST layer 604 also has (1) an STL caller 624 in each 10chain of each source GVM, (2) an STL module 626 in the IO chain of eachSVM, and (3) one or more encap processors 628.

For a data message that passes through a GVM's ingress or egressdatapath, the SI pre-processor 610 on this datapath performs severaloperations. It identifies the service chain for the data message andselects the service path for the identified service chain. Thepre-processor also identifies the network address for a first hopservice node in the selected service path and specifies the SMDattributes for the data message. The SMD attributes include in someembodiments the service chain identifier (SCI), the SPI and SI values,and the direction (e.g., forward or reverse) for processing the serviceoperations of the service chain. In some embodiments, the SPI valueidentifies the service path while the SI value specifies the number ofservice nodes.

After the SI pre-processor completes its operation, the STL caller 624in the same datapath calls the STL port proxy 620 to relay the SMDattributes and first hop's network address that the pre-processoridentified, so that the port proxy can forward the SMD attributesthrough the service plane to the first hop. The port proxy formats thedata message for forwarding to the first service node. In someembodiments, this formatting comprises replacing the original source anddestination MAC addresses in the data message with a service plane MACaddress that is associated with the source GVM 102 and the MAC addressof the first hop service node. This formatting also stores a set ofattributes for the data message that should be processed by otherservice transport layer modules (e.g., the other STL modules, etc.) onthe same host computer. These data message attributes include the SMDattributes as well as the original source and destination MAC addresses.

The STL port proxy 620 passes the formatted data message along with itsstored attributes to the software switch 120. Based on the destinationMAC address (i.e., the first hop MAC address) of the formatted datamessage, the software switch delivers the data message to the switchport associated with the first hop SVM. When the first hop is on thesame host computer as the port proxy 620, the data message is providedto the STL module 626 in the ingress IO chain of the first hop's servicenode on the same host computer. When the first hop is not on the samehost computer, the data message is encapsulated with an encapsulatingheader and forwarded to the next hop, as further described below.

Each hop's STL module 626 re-formats the data message by replacing theservice plane source MAC address and service plane destination MACaddress (i.e., its service node's MAC address) with the original sourceand destination MAC addresses of the data message. It then passes thisre-formatted data message with its accompanying SMD attributes to itshop's service proxy 614. This service proxy is in the IO chain of theingress datapath of the GVM. For purposes of preventing the illustrationin FIG. 6 from being overcomplicated with unnecessary detail, theingress and egress paths of each SVM in this example are combined inthis figure, unlike the ingress and egress paths 650 and 652 of the GVM102.

The service proxy 614 encapsulates the received data message with anencapsulating NSH header that stores the data message's SMD attributesand provides this encapsulated data message to its service node when theservice node can support NSH headers. When the service node is an SVM,the service proxy in some embodiments supplies the data messages and itsNSH header to the SVM's VNIC through a VNIC injection process, asfurther described below. When the service node cannot process NSHheaders, the service proxy 614 stores the SMD attributes into a legacyQinQ encapsulating header or a GRE encapsulating header, and then passesthe encapsulated data message to the VNIC of the SVM. These headers willbe further described below.

In some embodiments, the service proxy 614 of each service hopsegregates the service node for that hop from the service transportlayer. This segregation improves the security of both the SVM and theservice transport layer. It also allows the service proxy to ensure thatthe data messages that are provided to its SVM are formatted properly,which is especially important for legacy SVMs that do not support thenewer NSH format.

The service proxy 614 in some embodiments also performs livenessdetection signaling with its service node to ensure that the servicenode is operational. In some embodiments, the service proxy sends a datamessage with a liveness value to its service node at least once in eachrecurring time period. To do this, the service proxy sets and resets atimer to ensure that it has sent a liveness signal for each time periodto its service node. Each liveness value is accompanied with a livenesssequence number to allow the service proxy to keep track of livenessresponses provided by the SVM. Each time the service node replies to aliveness signal, it provides to the service proxy the same livenessvalue in a responsive data message in some embodiments or itscorresponding value in the responsive data message in other embodiments.Also, with each liveness responsive data message, the service nodeprovides the same sequence number in some embodiments, or an incrementedversion of the sequence number provided by the service proxy in otherembodiments.

As further described below, the service proxy of some embodimentspiggybacks some of its liveness detection signaling on each data messagethat it passes to its service node from the service forwarding plane.Each time that the service proxy sends a liveness signal to its servicenode, it resets its liveness timer. Each time the service node processesthe data message, it provides the processed data message back to theservice node with the responsive liveness value and associated sequencenumber (incremented in some embodiments, or non-incremented in otherembodiments, as mentioned above).

In some embodiments, the service proxy registers a liveness detectionfailure when the service node does not respond to its liveness signalwithin a particular time (e.g., within 0.3 seconds). After registeringtwo successive liveness detection failures, the service proxy in someembodiments notifies a local control plane (LCP) module executing on itshost the SVM has failed so that the LCP can notify a central controlplane (CCP) server. In response to such a notification, the CCP removesthe SVM and the service paths on which SVM resides from the forwardingand path selection rules in the data plane, and if needed, generatesadditional service paths for the failed SVM's associated service chain.Also, in some embodiments, the service proxy sends an in-band datamessage back to the source GVM to program its classifier to not selectthe service path on which the failed service node resides.

In some embodiments, the service proxy also performs flow programming atthe behest of its service node. This flow programming in someembodiments involves modifying how the source GVM's IO chain selectsservice chains, service paths, and/or forwards data message flows alongservice paths. In other embodiments, this flow programming involvesother modifications to how a data message flow is processed by theservice plane. Flow programming will be further described below.

Upon receiving a data message and its SMD attributes (in anencapsulating NSH header or some other encapsulating header), the SVMperforms its service operation. In some embodiments, the SVM usesmapping records that it receives from its service manager to map theSCI, SI and direction values in the SMD attributes to a service profile,and then maps this service profile to one of its rule sets, which itthen examines to identify one or more service rules to process. In someembodiments, each service rule has a rule identifier that is defined interms of data message attributes (e.g., five tuple attributes, which arethe source and destination IP address, source and destination portaddresses and the protocol). The SVM in some embodiments compares therule's identifier with the attributes of the data message to identify amatching rule. Upon identifying one or more matching rules, the SVM insome embodiments performs an action specified by the highest prioritymatching rule. For instance, a firewall SVM might specify that the datamessage should be allowed to pass, should be dropped and/or should beredirected.

Once the SVM has completed its service operation, the SVM forwards thedata message along its egress datapath. The service proxy in the egressdatapath's IO chain then captures this data message and for this datamessage, identifies the network address of the next hop in the servicepath. To do this, the service proxy in some embodiments decrements theSI value, and then uses this decremented value along with the SPI valuein the data message's stored attribute set to identify an exact matchforwarding rule that identifies a next hop network address. In someembodiments, the SVM can decrement the SI value. For such cases, theservice proxy in some embodiments can be configured not to decrement theSI value when its corresponding SVM decremented it.

In either configuration, the service proxy identifies the next hopnetwork address by using the appropriate SPI/SI values to identify thenext-hop forwarding rule applicable to the data message. When theproxy's service node is on multiple service paths, the proxy'sforwarding rule storage stores multiple exact match forwarding rulesthat can specify different next hop network addresses for differentSPI/SI values associated with different service paths. Assuming that thedecremented SI value is not zero, the next hop in the service path isanother service node. Hence, the proxy in some embodiments provides thenext hop's MAC address to the proxy's associated STL module 626 in theSVM's egress datapath. This module then re-formats the data message, byspecifying the SVM's MAC address and the next hop's MAC address as thesource and destination MAC addresses and storing the original source anddestination MAC addresses of the data message in the stored set ofattributes stored for the data message. The STL module 626 then forwardthe data message along the egress path, where it reaches the softwareswitch, which then has to forward the data message and its storedattributes to the next hop service node.

When the next hop is on the same host computer, the software switchpasses the data message and its attributes to the port that connects tothe STL module of the next hop's service node, as described above. Onthe other hand, when the next hop service node is on another hostcomputer, the software switch provides data message to the uplink portthat connects to the VTEP (VXLAN Tunnel Endpoint) that communicatesthrough an overlay network tunnel with a VTEP on the other hostcomputer. An encap processor 628 then captures this data message alongthe egress path of this port, defines an encapsulating overlay headerfor this data message and encapsulates the data message with thisoverlay header. In some embodiments, the overlay header is a singleheader that stores both SMD and STL attributes. For instance, in someembodiments, the overlay header is a Geneve header that stores the SMDand STL attributes in one or more TLVs.

As mentioned above, the SMD attributes in some embodiments include theSCI value, the SPI value, the SI value, and the service direction. Also,in some embodiments, the STL attributes includes the original L2 sourceMAC address, the original L2 destination MAC address, the data messagedirection, and the service-plane source MAC address of the source GVM.In some embodiments, the service direction and the service-plane sourceMAC address are already part of the SMD attributes. The servicetransport layer in some embodiments needs these attributes with eachprocessed data message, in order to recreate the original data messageand later at the end of the service-path, to return the data message tothe original host to resume along its datapath.

When the encapsulated data message is received at the next hop's hostcomputer, the data message is captured by the encap processor 628 of thesoftware switch's downlink port that connects to the VTEP that receivedthe data message from the prior hop's VTEP. This encap processor removesthe encapsulation header from the data message and stores the STL andSMD attributes as the set of attributes of the data message. It thenpasses the decapsulated message to the downlink port, which then passesit to the software switch to forward to the next hop's switch port. Fromthere the data message is processed by the STL module and service proxybefore reaching the service node, as described above.

When the service proxy determines that the decremented SI value is zero,the service proxy matches the decremented SI value and the embedded SPIvalue with a rule that directs the service proxy to identify the nexthop as the service plane MAC address of the source GVM. In someembodiments, this determination is not specified by a forwarding entryof a forwarding table, but rather is hard coded into the logic of theservice proxy. Hence, when the SI value is zero, the proxy provides thesource GVM's service plane MAC address to its associated STL module 626to use to forward the data message back to the GVM's host computer. TheSTL module then defines the message's destination MAC (DMAC) address asthe source GVM's service plane MAC address while defining the message'ssource MAC (SMAC) address as the service plane MAC address associatedwith its service node (e.g., the service plane MAC of the softwareswitch's port associated with the service node). It also stores theoriginal SMAC and DMAC of the data message in the attribute set of thedata message.

The STL module then passes the formatted data message and its attributesalong the egress path, where it reaches it associated software switchport. The software switch then passes this message to its uplink port.The encap processor 628 of this port then captures this data message,defines an encapsulating overlay header for this data message andencapsulates the data message with this overlay header. As mentionedabove, this overlay header is a Geneve header that stores the SMD andSTL attributes in one or more TLVs. This encapsulated data message thentraverses the overlay network to reach the source GVM's host computer,where this data message is decapsulated by the downlink port's encapprocessor, and is then provided to the software switch, which thenforwards it to the port proxy.

Once the port proxy 620 receives the decapsulated data message, itidentifies the GVM associated with this data message from the originalsource MAC address that is now part of the decapsulated data message'sstored attributes. In some embodiments, the port proxy has a record thatmaps the original source MAC address and service direction in the SMDattributes of a received data to a GVM on its host (e.g., to a softwareswitch port associated with a guest forwarding plane and a GVM on itshost). The port proxy then formats the data message to include itsoriginal SMAC and DMAC and provides the data message back to the sourceGVM's IO chain. The SI post-processor 612 in this IO chain thenprocesses this data message, before returning this data message to theegress datapath of the GVM. The operations of this post-processor willbe further described below.

One of ordinary skill will realize that the service insertion layer andservice transport layer in other embodiments are implemented differentlythan the exemplary implementations described above. For instance,instead of using an L2 overlay (L2 transport layer) that relies on MACaddresses to traverse the different service hops, other embodiments usean L3 overlay (L3 transport layer) that uses L3 and/or L4 networkaddresses to identify successive service hops. Also, the above-describedservice insertion and/or transport modules can be configured to operatedifferently.

A more detailed example of the operations of the service insertion andservice transport layers will now be described by reference to FIGS.7-19. FIG. 7 illustrates a process 700 performed by the SI pre-processor610 and STL caller 624 of some embodiments. This process is describedbelow by reference to the data flow example illustrated in FIG. 8. Theprocess 700 starts when the SI pre-processor 610 is called to analyze adata message that is sent along the ingress or egress datapath of a GVM.

As shown, the process 700 initially determines (at 705) whether thepre-processor 610 has previously selected a service chain and a servicepath for the data message's flow and stored the SMD attributes for theselected service chain and path. In some embodiments, the process 700makes this determination by using the data message's attributes (e.g.,its five tuple attributes) to try to identify a record for the message'sflow in a connection tracker that stores records of message flows forwhich service chains and paths were previously selected, and SMDattributes were previously stored for these chains and paths in theconnection tracker records.

FIG. 8 illustrates the pre-processor 610 receiving a data message 802along the egress datapath of the GVM 102. It also shows thepre-processor initially checking a connection tracking storage 804 totry to find a connection record that has a flow identifier (e.g., afive-tuple identifier) that matches a set of attributes (e.g., fivetuple attributes) of the received data message. In this example, thepre-processor 610 cannot find such a connection record as the receiveddata message is the first data message for its flow.

When the process 700 determines (at 705) that the connection storage 804has a connection record that matches the received data message, theprocess retrieves (at 710) the SMD attributes from this record, or fromanother record referenced by the matching connection record. The SMDattributes in some embodiments include the SCI, SPI, SI and directionvalues. From 710, the process transitions to 740, which will bedescribed below.

On the other hand, when the process 700 determines (at 705) that theconnection storage 804 does not have a connection record that matchesthe received data message, the process performs (at 715) aclassification operation that tries to match the data message to aservice insertion rule in a SI rule storage, which is illustrated inFIG. 8 as storage 806. In some embodiments, the SI rule storage 806stores service insertion rules 822 that have rule identifiers defined interms of one or more data message flow attributes (e.g., one or more ofthe five tuple attributes or portions thereof). Each service rule alsospecifies a SCI that identifies a service chain that is applicable todata message flows that match the rule identifier of the service rule.

At 720, the process determines whether the classification operationmatches the data message's attributes to the rule identifier of aservice insertion rule that requires a service chain to be performed onthe data message. When the classification operation does not identify aservice insertion rule that requires a service chain to be performed onthe data message, the process 700 ends. In some embodiments, the SI rulestorage 806 has a default low priority rule that matches any datamessage when the data message's attributes do not match any higherpriority SI rule, and this default low priority rule specifies that noservice chain has been defined for the data message's flow. No servicechain is defined for a data message flow in some embodiments when noservice operations needs to be performed on the data message flow.

On the other hand, when the classification operation matches the datamessage's attributes to the rule identifier of a service insertion rulethat requires a service chain to be performed on the data message, theprocess 700 performs (725) a path selection operation to select aservice path for the service chain specified by the service insertionrule identified at 715. As shown in FIG. 8, the pre-processor 610performs a path-selection operation by examining a path storage table808 that identifies one or more service paths for each service chainidentifier.

Each service path is specified in terms of its SPI value. When multipleservice paths are specified for a service chain, the path storage 808stores for each service chain a set of selection metrics 820 forselecting one SPI from the available SPIs. Different embodiments usedifferent selection metrics. For instance, some embodiments use aselection metric that costs a service path based on the number of hostson which the service nodes of the service path execute. In otherembodiments, these selection metrics are weight values that allow thepre-processor to select SPIs for a service chain in a load balancedmanner that is dictated by these weight values. For instance, in someembodiments, these weight values are generated by a central controlplane based on the load on each of the service nodes in the service pathand/or based on other costs (such as number of hosts traversed by theservice path, etc.).

In some of these embodiments, the pre-processor maintains a record ofprevious selections that it has made for a particular service chain, andselects subsequent service paths based on these previous selections. Forexample, for four service paths, the weight values might be 1, 2, 2, 1,which specify that on six successive SPI selections for a service chain,the first SPI should be selected once, the second and third SPIs shouldthen be selected twice each, and the fourth SPI should be selected one.The next SPI selection for this service chain will then select the firstSPI, as the selection mechanism is round robin.

In other embodiments, the weight values are associated with a numericalrange (e.g., a range of hash values) and a number is randomly ordeterministically generated for each data message flow to map the datamessage flow to a numerical range and thereby to its associated SPI. Instill other embodiments, the hosts LCP selects one service path for eachservice chain identifier from the pool of available service paths, andhence stores just one SPI for each SCI in the path table 808. The LCP inthese embodiments selects the service path for each service chain basedon costs (such as the number of hosts traversed by each service pathand/or the load on the service nodes of the service paths).

After identifying a service path for the identified service chain, theprocess 700 next identifies (at 730) the network address for the firsthop of the selected service path. In some embodiments, the MAC addressfor this hop is stored in the same record as the selected path's SPI.Hence, in these embodiments, this MAC address is retrieved from the pathselection storage 808 with the selected SPI. In other embodiments, thepre-processor retrieves the first hop's MAC address from an exact matchforwarding table 810 that stores next hop network addresses forassociated pairs of SPI/SI values, as shown in FIG. 8. In someembodiments, the initial SI values for the service chains are stored inthe SI rules of the SI rule storage 806, while in other embodiments,these initial SI values are stored with the SPI values in that pathtable 808.

At 735, the process 700 specifies the SMD attributes for the datamessage, and associates these attributes with the data message. Asmentioned above, the SMD attributes include in some embodiments the SCI,the SPI, SI and direction values. The service directions for servicepaths are stored with the SPI values in the path table 808 as thedirections through the service chains are dependent on the servicepaths. Also, as mentioned below, a service chain in some embodiments hasto be performed in a forward direction for data messages from a firstGVM to a second GVM, and then in the reverse direction for data messagesfrom the second GVM to the first GVM. For such service chains, thepre-processor 610 selects both the service path for the forwarddirection and the service path for the reverse direction when itprocesses the first data message in the flow from the first GVM to thesecond GVM.

After the SI pre-processor completes its operation, the STL caller 624in the same datapath calls (at 740) the STL port proxy 620 to relay theSMD attributes and first hop's network address that the pre-processoridentified, so that the port proxy can forward the SMD attributesthrough the service plane to the first hop. The operation of the portproxy 620 as well as other modules in the service insertion layers andservice transport layers will be described by reference to FIGS. 9-19.These figures describe an example of processing the data message fromGVM 102 through a service path that includes the SVM 106, then SVM 108and then SVM 110.

In these figures, each GVM is a compute machine of a tenant in amulti-tenant datacenter, and connects to the software switch through aswitch port that is associated with a guest VNI (GVNI) of the tenant.Also, in these figures, each SVM is a service machine for processing theGVM message traffic, and connects to the software switch through aswitch port that is associated with a service VNI (SVNI) of the tenant.As mentioned above and further described below, some embodiments use theGVNI for performing the guest logical forwarding operations (i.e., forestablishing a guest logical forwarding element, e.g., a logical switchor router, or a guest logical network) for the tenant, while using theSVNI for performing the service logical forwarding operations for thetenant (i.e., for establishing a service logical forwarding element,e.g., a logical switch or router, or a service logical network).

Both of these logical network identifiers (i.e., the GVNI and SVNI) aregenerated for the tenant by the management or control plane in someembodiments. The management or control plane of some embodimentsgenerates different GVNIs and SVNIs for different tenants such that notwo tenants have the same GVNI or SVNI. In some embodiments, each SVM isdedicated to one tenant, while in other embodiments, an SVM can be usedby multiple tenants. In the multi-tenant situation, each SVM can connectto different ports of different service planes (e.g., different logicalswitches) for different tenants.

As shown in FIG. 9, the port proxy 620 formats the data message forforwarding to the first service node, by replacing the original sourceand destination MAC addresses in the data message with a service planeMAC address that is associated with the source GVM 102 and the MACaddress of the first hop service node. This operation is depicted asoperation 1005 in the process 1000 of FIG. 10. This process 1000 is aprocess that the port proxy 620 or STL module 626 starts whenever an SImodule (such as an SI pre-processor 610 or a SI proxy 614) is doneprocessing a data message.

In this process 1000, the port proxy also adds (at 1010) the originalsource and destination MAC addresses of the data message to the set ofattributes for the data message that should be processed by otherservice transport layer modules (e.g., the vswitch, other STL modules,the encap processor, etc.) on the same host computer. The reformatteddata message 902 and the augmented attributed set 904 are depicted inFIG. 9.

After reformatting the data message and augmenting its attribute set,the port proxy 620 passes (at 1015) the formatted data message alongwith its stored attribute set along its egress path where it reaches thesoftware switch 120. Based on the destination MAC address (e.g., thefirst hop MAC address) of the formatted data message, the softwareswitch determines (at 1020) whether the next hop's port is local. Thisis the case for the example illustrated in FIG. 9. Hence, the softwareswitch delivers (at 1025) the data message to the switch port associatedwith the first hop SVM 106. This port then sends the data message alongthe SVM's ingress path, where the data message 902 and its augmentedattribute set 904 is identified by the STL module 626 through a functioncall of the ingress IO chain of the first hop's SVM, as shown in FIG. 9.

This STL module 626 then re-formats (at 1030) the data message byreplacing the GVM's service plane MAC address and the first hop MACaddress (i.e., the MAC address of SVM 106) with the original source anddestination MAC addresses of the data message, which it retrieves fromthe augmented attribute set 904. In retrieving the original SMAC andDMAC addresses, the STL module 626 modifies the data message's attributeset. The reformatted data message 906 and the modified attributed set908 are depicted in FIG. 9. The STL module then passes this re-formatteddata message with its accompanying SMD attributes along the SVM'singress path, where it is next processed by this hop's ingress serviceproxy 614.

FIG. 11 illustrates a process 1100 that the service proxy 614 performsin some embodiments each time it receives a data message traversingalong the ingress path of a service node. As shown, the service proxyinitially makes (at 1105) a copy of the data message if necessary. Forinstance, in some embodiments, the service node only needs to receive acopy of the data message to perform its operations. One example of sucha service node would a monitoring SVM that needs to obtain a datamessage copy for its message monitoring or mirroring operation.

In these embodiments, the service proxy copies the data messages andperforms the remaining operations 1110-1125 with respect to this copy,while passing the original data message to the next service hop or backto the source GVM. To forward the original data message to the nextservice hop or back to the GVM, the service proxy has to perform anext-hop lookup based on the SPI/SI values and then provide the next-hopaddress (e.g., the next service hop's address or the service plane MACof the source GVM) to the STL module to forward. These look up andforwarding operations are similar to those described below by referenceto FIGS. 15-17.

Next, at 1110, the service proxy sets a liveness attribute in the storedSMD attribute set of the data message (which, in some embodiments, mightbe the data message copy at this point). This liveness attribute is avalue that directs the service node to provide a responsive livenessvalue (the same value or related value) with the data message once ithas processed the data message. With this liveness attribute, theservice proxy also provides a sequence number, which the service nodehas to return, or increment and then return, with the responsiveliveness value, as described above.

At 1115, the service proxy formats the data message, if necessary, toput it in a form that can be processed by the service node. Forinstance, when the service node does not know the current next hop MACthat is set as the destination MAC of the data message, the serviceproxy changes the destination MAC of the message to a destination MACassociated with the service node.

After formatting the data message to sanitize it for forwarding to theservice node, the service proxy 614 encapsulates (at 1120) the datamessage with one of three encapsulation headers that it can beconfigured to use, and passes (at 1125) the encapsulated message alongthe service node's ingress path so that it can be forwarded to theservice node. FIG. 9 illustrates the encapsulated data message 920passing from the service proxy to the SVM 106 with a native NSHencapsulation header. As shown, the encapsulating header 922 includesthe service chain identifier, the service index, service chain directionand liveness signal.

FIG. 12 illustrates the three encapsulation headers of some embodiments,which are (1) a native NSH encapsulation header 1205 for a service nodethat support NSH, (2) a GRE encapsulation header 1210 for a legacyservice node that does not support NSH, and (3) a QinQ encapsulationheader 1215 for a legacy service node that does not support NSH. Thenative NSH header stores the service metadata in a format that isdescribed below by reference to FIGS. 21 and 22. The GRE header formatwill be described further below by reference to FIG. 25-26. In both theGRE and QinQ formats a portion of the service metadata is stored in theGRE and QinQ header fields, but the service metadata cannot be stored asrichly as it is stored in the native NSH header. The QinQ header is usedfor simple legacy service nodes that do not need much service metadata,e.g., just need service chain identifier and service direction, orservice chain identifier and service index. This service metadata isstored in the VLAN header field of the QinQ header.

In addition to the three different types of encapsulating headers 1205,1210 and 1215, FIG. 12 also illustrates a vmxnet3 paravirtualized NIC1240 of an SVM of some embodiments. As shown, this NIC can provide theencapsulated data message to a poll mode driver 1202 of a DPDK driver1204 of the SVM, or to an interrupt mode driver 1204. Specifically, thevmxnet3 paravirtualized NIC can be configured to operate in differentmodes of operation depending on which driver is used inside the SVM. Thepoll mode driver 1202 can be viewed as the back end of the DPDK (dataplane development kit) driver 1206. The poll mode driver regularly pollsthe VNIC for data messages to retrieve, while the VNIC generatesinterrupts to cause the interrupt-based driver 1204 to retrieve the datamessages.

The poll mode driver passes a data message to the DPDK driver 1206,which then passes it to the message processing module in the user spacewhen a flow is initially received. The interrupt-based driver 1204, onthe other hand, provides the data message to the message processingmodule 1212 either in the kernel or in the user space. The messageprocessing module of the SVM then decapsulates the encapsulated datamessage and performs the service operation of the SVM. In someembodiments, different SVMs perform different service operations basedon the SCI, SI and service direction values that they receive with adata message.

FIG. 13 illustrates one exemplary process 1300 that an SVM performs insome embodiments each time it receives a data message to process from aservice proxy. In other embodiments, an SVM can use the SCI, SI andservice direction values differently to perform its operations. Asshown, the process 100 initially (at 1305) removes the encapsulatingheader and from it retrieves the SCI, SI, direction and livenessparameters. The process then uses (at 1310) mapping records that itreceives from its service manager to map the SCI, SI and directionvalues to a service profile, and then maps (at 1315) this serviceprofile to one of its rule sets, which it then examines to identify (at1320) one or more service rules to process.

FIG. 14 shows a first mapping table 1405 of the SVM. As shown, eachrecord in this table maps the SCI, SI and direction values to a serviceprofile. This figure also shows a second mapping table 1410 of the SVM,and this table maps a service profile to a rule set identifier thatidentifies several rules in a service rule table 1415. As indicated inFIG. 14, a service insertion manager in some embodiments provides therecords of the first table 1405 (e.g., an SI network manager providesthese records to a service manager of the SVM, which then provides themto the SVM), while the service manager of the SVM provides the recordsfor the second and third tables 1410 and 1415. In some embodiments,these two service managers are two different management planesadministered by two different entities, e.g., a datacenter administratorand a third-party administrator, or a tenant administrator and adatacenter administrator.

In some embodiments, each service rule 1420 in the service rule table145 has a rule identifier that is defined in terms of data messageattributes (e.g., five tuple attributes). The SVM compares (at 1320) arule's identifier to the attributes of the data message to identify amatching rule. Upon identifying one or more matching rules, the SVM insome embodiments performs (at 1325) an action specified by the highestpriority matching rule. For instance, a firewall SVM might specify thatthe data message should be allowed to pass, should be dropped and/orshould be redirected.

Once the SVM has completed its service operation, the SVM encapsulates(at 1330) the data message with an encapsulating header, assuming thatthe service operation does not result in the dropping of the datamessage. This encapsulating header has the same format (e.g., is an NSHheader, GRE header, or QinQ header) as the data message that the SVMreceived. In this encapsulating header, the SVM in some embodiments sets(1) a liveness value to respond to the service proxy's liveness valueand (2) the appropriate sequence number (e.g., unadjusted or incrementedsequence number) for the liveness value.

In some embodiments, some service nodes are configured to decrement theSI values that they receive, while other service nodes are notconfigured to decrement the SI values. If the service node is configuredto decrement the SI value, it decrements the SI value before insertingthe decremented SI value in the encapsulating header at 1330. The SVM insome embodiments also sets the SMD attributes (SCI, SI and servicedirection) in the encapsulating header, while in other embodiments, theservice proxy in the egress path retrieves these values from an earlierrecord that the service proxy created before passing the data message tothe SVM.

In some embodiments, the SVM can also set flow programming attribute(s)in the encapsulating header to direct the service proxy to modify theservice processing of the data message's flow. This flow programmingwill be further described below. After encapsulating the data message,the SVM forwards the data message along its egress path. FIG. 15illustrates an example of SVM 106 returning the encapsulated datamessage 1502 with the SMD and liveness attributes in its encapsulatingheader 1504.

FIG. 16 illustrates a process 1600 that the service proxy 614 performsin some embodiments each time it receives a data message traversingalong the egress path of its service node. As shown, the service proxyin some embodiments initially (at 1605) removes the encapsulation headerfrom the data message, removes the SMD attributes from this header, andstores these attributes in an attribute set that it creates for the datamessage. In some embodiments, the service proxy retrieves (at 1605) someor all of the SMD attributes (e.g., the SPI value, the service plane MACaddress of the source GVM) for the data message from a previous recordthat the service proxy created before giving the data message to theservice node along the ingress path. FIG. 15 illustrates an example ofthe attribute set 1506 that the service proxy 614 creates for thedecapsulated data message 1507.

Next, 1610, the process resets the liveness timer (e.g., a timer thatexpires every 0.25 seconds) that it maintains to account for theliveness value that it has received from the service node, whichsignifies that this node is still operational. With this liveness value,the service proxy receives from the service node a sequence number,which the process validates to ensure that it is the next liveness valuethat needs to be received.

At 1615, the process determines whether the SVM specified any flowprogramming attribute(s), which require the service proxy to direct theSI post processor 612 for the source GVM to perform flow programming bysending to the post processor 612 in-band data messages. In someembodiments, the service proxy sends an in-band flow programming controlsignal with another data message that it generates to send back to thesource GVM, where it will be intercepted by its post processor 612.

When the source GVM receives the data message with the flow programmingcontrol signal, its post processor can uniquely identify the datamessage flow to which it applies by using a flow identifier that isunique to this flow. As further described below, this flow identifier isderived partially based on a unique identifier of the source GVM. Theunique flow identifier also allows other service plane modules, such asthe service nodes, service proxies and STL modules, to uniquely identifyeach data message flow. This unique flow identifier in some embodimentsis part of the SMD attributes that are passed between the service hopsof a service path and passed back to the source GVM.

In some embodiments, however, the service proxy sends the in-band flowprogramming control signal with the current data message that it isprocessing. In some of these embodiments, the service proxy does thisonly when its associated service node is the last hop service node ofthe service path, while in other embodiments it does this even when itsservice node is not the last hop service node. When its service node isnot the last hop service node of the service path, the service proxyembeds the flow programming in the SMD attributes of the data message,which in some embodiments eventually get forwarded to the source GVM'sSI post processor as part of the data message encapsulation header whenthe last hop service is performed. Even in this situation, the serviceproxy of the last hop in other embodiments sends the flow programmingsignal as a separate message.

The flow programming signals will be further described below byreference to FIG. 20. Also, as further described below, the serviceproxy also sends flow programming signals back to the source GVM when itdetects that its service node has failed so that the classifier at thesource GVM can select another service path for the current data messageflow, as well as other data message flows. In such a situation, theservice proxy also notifies the LCP on its host computer, so that theLCP can notify the CCP and the CCP, in turn, can modify the servicepaths specified for service chains that use the failed service node.

At 1620, the process 1600 determines whether its service node specifiedthat the data message should be dropped. If so, the process drops thedata message and then ends. Otherwise, assuming the data message shouldnot be dropped and should continue along its service path, the serviceproxy in some embodiments decrements (at 1625) the SI value in case theservice node has not decremented the SI value, and then uses (at 1630)this decremented value along with the SPI value in the data message'sstored attribute set to identify an exact match forwarding rule thatidentifies a next hop network address. When the proxy's service node ison multiple service paths, the proxy's forwarding rule storage storesmultiple exact match forwarding rules that can specify different nexthop network addresses for different SPI/SI values.

When the decremented SI value is zero, the service proxy in someembodiments that matches the decremented SI value and the embedded SPIvalue with a rule that directs the service proxy to identify the nexthop as the service plane MAC address of the source GVM. This rule insome embodiments does not provide a MAC address, but rather refers tothe service plane MAC address that is part of the SMD attribute setstored for the data message. In some embodiments, this instructions forreturning the data message to the service plane MAC address of thesource GVM when the SI value is zero is not specified by a forwardingentry of a forwarding table, but rather is hard coded into the logic ofthe service proxy.

At 1630, the service proxy stores the next hop network address (e.g.,MAC address) in the attribute set that is stored for the data message.FIG. 15 illustrates an example of the service proxy 614 storing the nexthop MAC address associated with the next service node in the attributeset 1506 of the decapsulated data message 1507. After identifying thenext hop network address, the service proxy returns (at 1635) the datamessage to the egress path of its service node, and the process 1600ends.

Once the service proxy returns the data message to the service node'segress path, the STL module 626 receives this data message and commencesthe process 1000 of FIG. 10. The STL module 626 performs the first threeoperations 1005-1015 of this process each time it receives a datamessage from a service insertion layer. Specifically, the STL moduleformats (at 1005) the data message for forwarding to the next hopservice node, by replacing the original source and destination MACaddresses in the data message with a service plane MAC addresses of thecurrent service hop and the next service hop (i.e., the hop1mac andhop2mac addresses in the example illustrated in FIG. 15).

At 1010, the STL module also adds the original source and destinationMAC addresses of the data message to the set of attributes for the datamessage that should be processed by other service transport layermodules (e.g., the vswitch, the encap processor, etc.) on the same hostcomputer. The reformatted data message 1508 and the augmented attributedset 1510 are depicted in FIG. 15. After reformatting the data messageand augmenting its attribute set, the STL module 626 passes (at 1015)the formatted data message along the egress path, where it next reachesthe software switch 120.

Based on the destination MAC address (i.e., the next hop MAC address) ofthe formatted data message, the software switch determines (at 1020)that the next hop's port is not local. Hence, the software switchprovides (at 1035) the data message to the uplink port 1550 thatconnects to a VTEP1 that communicates through an overlay network tunnelwith a VTEP2 on host 114, as illustrated in the example of FIG. 15. Asshown, an STL encap processor 628 along the egress path of this uplinkport (at 1040) receives this data message (e.g., is called as one of thehooks specified for the uplink port), defines an encapsulating overlayheader 1540 for this data message and encapsulates the data message withthis overlay header.

In some embodiments, the overlay header is a Geneve header that storesthe SMD and STL attributes in one or more of its TLVs. As mentionedabove, the SMD attributes in some embodiments include the SCI value, theSPI value, the SI value, and the service direction. Also, in someembodiments, the STL attributes includes the original L2 source MACaddress and the original L2 destination MAC address. FIG. 15 illustratesan example of this encapsulating header, which will be further describedbelow by reference to FIG. 28.

When the encapsulated data message is received at the next hop's hostcomputer 114, the data message is captured by the STL encap processor628 of (e.g., defined as a hook for) a downlink port 1552 that connectsto the VTEP connecting through the overlay network tunnel to the priorhop's VTEP. FIG. 17 illustrates a process 1700 started by an encapprocessor 628 on a next hop computer that receives an encapsulated datamessage that needs to be processed by an SVM executing on its computer.

As shown, this encap processor removes (at 1705) the encapsulationheader from the data message, and stores (at 1705) the STL and SMDattributes as the associated set of attributes of the data message. Itthen passes (at 1710) the decapsulated message to the downlink port,which then passes it to the software switch to forward (at 1715) to itsport that is connected to the next hop SVM (i.e., that is associatedwith the destination MAC address). This port then passes the datamessage 1508 and the attribute set 1510 to the ingress path of the nexthop SVM, as shown in the example of FIG. 15 for the SVM 108.

The STL module 626 on this ingress path then re-formats (at 1720) thedata message by replacing the previous and current hop service plane MACaddress (i.e., the hop1mac and hop2mac) with the original source anddestination MAC addresses of the data message, which it retrieves fromthe data message attribute set. In retrieving the original SMAC and DMACaddresses, the STL module 626 modifies the data message's attribute set.The reformatted data message 1530 and the modified attributed set 1532are depicted in FIG. 15. The STL module then passes this re-formatteddata message with its accompanying SMD attributes along the SVM'singress path, where it is next processed by this hop's ingress serviceproxy 614.

The operation of this service proxy is as described above by referenceto FIGS. 9 and 11. FIG. 15 shows the service proxy of SVM 108 on host114 passing an encapsulated data message to the SVM. The encapsulatingheader of this data message is supported by the SVM 108 and stores theSCI, SI, service direction and liveness values. In some embodiments, theSVMs that are part of the same service path support differentencapsulating headers. In some of these embodiments, the service proxiesalong a service path can encapsulate the data message with differentencapsulating headers before passing the data message to theirassociated SVMs. For instance, in one case, the first hop service proxypasses to the SVM 106 the data message with an NSH encapsulating header,while the second hop service proxy passes to the SVM 108 the datamessage with a QinQ encapsulating header.

Once the SVM 108 performs its service operation on the data message(e.g., per the process 1300 of FIG. 13), the SVM sends the processeddata message along its egress data path, as shown in FIG. 18. As shown,the service proxy then identifies the MAC address of the next servicehop and adds this MAC address to the stored attribute set for the datamessage. At this point, the next hop is the third service hop, whichcorresponds to the SVM 110. This proxy identifies this MAC bydecrementing the SI value (when the SVM 108 did not decrement the SIvalue) and then using the embedded SPI value and decremented SI value tolookup a forwarding rule that provides the next hop's MAC address. TheSTL module in this egress path then replaces the original SMAC and DMACin the data message with the current hop and next hop MAC addresses(i.e., the hop2mac and the hop3mac in the example of FIG. 18), storesthe original SMAC and DMAC in the stored attribute set of the datamessage, and then passes the data message along the egress path where itis received by the software switch 122.

The software switch then determines that the next hop is associated withits uplink port 1552, and hence passes the data message to this port. Asshown in FIG. 18, the encap processor 628 on the egress path of thisport (e.g., specified as a hook on this egress path) then encapsulatesthe data message with a Geneve header that stores the SMD and STLattributes in one or more of TLVs and specifies that the data message istraversing from this port's associated VTEP2 to VTEP3 that is associatedwith port 1554 of host 116.

The STL encap processor 628 in the ingress path of port 1554 thenremoves the encapsulation header from the data message and stores theSTL and SMD attributes as the associated set of attributes of the datamessage. It then passes the decapsulated message to the port 1554, whichthen passes it to the software switch 124 to forward to its portconnected to the next hop SVM 110 (i.e., to its port associated with theservice plane DMAC). This port then passes the data message andattribute set to the ingress path of this SVM, as shown in FIG. 18.

The STL module 626 in this ingress path replaces the previous andcurrent hop service plane MAC address (i.e., the hop2mac and hop3mac)with the original source and destination MAC addresses of the datamessage, which it retrieves from the data message attribute set. The STLmodule 626 also modifies the data message's attribute set by removingthe original SMAC and DMAC addresses, and then passes the re-formatteddata message with its accompanying SMD attributes along the SVM'singress path for this hop's ingress service proxy 614 to process. Thisservice proxy passes to the SVM 110 an encapsulated data message with anencapsulating header supported by the SVM 110 and storing the SCI, SI,service direction and liveness values.

Once the SVM 110 performs its service operation on this data message(e.g., per the process 1300 of FIG. 13), the SVM sends the processeddata message along its egress data path, as shown in FIG. 19. Theservice proxy decrements the SI value when assuming that the SVM 110 hasnot done so already. In this example, the decremented SI value is nowzero. In some embodiments, the service proxy then matches this SI valueand the SPI value to a rule identifier of a forwarding rule thatspecifies that it should select the service plane MAC (spmac) of thesource GVM as the next hop MAC address. In other embodiments, thehardcoded logic of the service proxy directs it to identify the serviceplane MAC of the source GVM as the next hop MAC. In either case, theservice proxy adds the source GVM's service plane MAC to the attributeset of the data message.

The STL module next replaces the original SMAC and DMAC in the datamessage with the third hop MAC address and the source GVM's serviceplane MAC, stores the original SMAC and DMAC in the stored attribute setof the data message, and then passes the data message to its softwareswitch 124. The software switch then determines that the next hop isassociated with its port 1554, and hence passes the data message to thisport. As shown in FIG. 19, the encap processor 628 on the egress path ofthis port then encapsulates the data message with a Geneve header thatstores the SMD and STL attributes in one or more TLVs and specifies thatthe data message is traversing from this port's associated VTEP3 toVTEP1 that is associated with port 1550 of host 112.

The STL encap processor 628 in the ingress path of port 1550 thenremoves the encapsulation header from the data message and stores theSTL and SMD attributes as the associated set of attributes of the datamessage. It then passes the decapsulated message to the port 1550, whichthen passes it to the software switch 120 to forward to its portconnected to the port proxy 620. This port then passes the data messageand attribute set to the port proxy 620, as shown in FIG. 19.

The port proxy 620 then replaces the previous and current hop serviceplane MAC address (i.e., the hop3mac and spmac) with the original sourceand destination MAC addresses of the data message, which it retrievesfrom the data message attribute set. The port proxy 620 also modifiesthe data message's attribute set to remove the original SMAC and DMAC,and then passes this re-formatted data message with its accompanying SMDattributes back to the STL caller 624 that called it in the first place.In some embodiments, the port proxy uses a connection record that itcreated when the STL caller originally called it, to identify the STLcaller to call back. In other embodiments, the port proxy uses a mappingtable that maps each service plane MAC with a GVM's STL caller. Themapping table in some embodiments has records that associate serviceplane MACs and service directions with guest forwarding plane portidentifiers associated with the GVMs.

Once called, the STL caller passes the data message along the egresspath of GVM 102, where it will next be forwarded to the SIpost-processor 612. FIG. 20 illustrates a process 2000 that the SIpost-processor 612 performs in some embodiments. The post-processorperforms this process 2000 each time it receives a data message that ispassed to it along a GVM's IO chain. As shown, the post processor 612 insome embodiments initially determines (at 2005) whether it needs toexamine the received data message for SI post processing. This isbecause as a module along a GVM's IO chain, the post processor will getcalled for all data message flows that pass along this IO chain and someof these data message might not match an SI rule that requires serviceinsertion operations to be performed on them. In some embodiments, theprocess 2000 determines (at 2005) whether it needs to process the datamessage by determining whether the data message has associated servicemetadata. If not, the process transitions to 2020, which will bedescribed below.

When the SI post processor 612 determines that it needs to process thedata message, the process determines (at 2010) whether the SMD metadataassociated with the data message specifies a flow programming tag thatrequires the post processor to perform a flow programming operation. Insome embodiments, such a flow programming tag would be specified in thedata message's SMD attributes by a service node to change the servicepath processing at the source GVM, or by a service proxy for the samereason when it detects failure of its service node. When the flowprogramming tag does not specify any flow programming, the processtransitions to 2020, which will be described below.

Otherwise, when the flow programming tag specifies a flow programmingoperation, the process 2000 performs this operation, and thentransitions to 2020. The flow programming operation entails in someembodiments modifying the connection record in the connection trackingstorage 804 to specify the desired operation and/or SMD attributes(e.g., allow, drop, etc.) for the data message's flow. The postprocessor's writing to the connection tracker 804 is depicted in FIG.19. As mentioned above and further described below, the SMD metadata forthe processed data message includes a flow identifier that uniquelyidentifies the data message's flow by being at least partially derivedfrom the unique service plane identifier of the source GVM. The postprocessor 612 uses this flow identifier to match the data message's flowin the connection tracker in some embodiments.

In some embodiments, the flow programming tag can specify the followingoperations (1) NONE when no action is required (which causes no flowprogramming operation to be performed), (2) DROP when no further datamessages of this flow should be forwarded along the service chain andinstead should be dropped at the source GVM, (3) ACCEPT when no furtherdata messages of this flow should be forwarded along the service chainand instead the flow should be accepted at the source GVM. In someembodiments, the flow programming tag can also specify DROP MESSAGE. TheDROP MESSAGE is used when the service node needs to communicate with theproxy (e.g. to respond to a ping request) and wants the user datamessage (if any) to be dropped, even though no flow programming at thesource is desired.

In some embodiments, an additional action is available for the serviceproxies to internally communicate failure of their SVMs. This actionwould direct the SI post processor in some embodiments to select anotherservice path (e.g., another SPI) for the data message's flow. Thisaction in some embodiments is carried in-band with a user data messageby setting an appropriate metadata field in some embodiments. Forinstance, as further described below, the service proxies communicatewith the post processor of the source GVM through OAM (Operation,Administration, and Maintenance) metadata of the NSH attributes throughin-band data message traffic over the data plane. Given that by designflow programming actions are affected by signaling delays and aresubject to loss, an SVM or service proxy might still see data messagesbelonging to a flow that was expected to be dropped, accepted orre-directed at the source for some time after communicating the flowprogramming action to the proxy. In this case, the service plane shouldcontinue set action to drop, allow or redirect at the source.

The process 2000 transitions to 2020 after completing the flowprogramming operation. It also transitions to 2020 when it determines(at 2005) that no SI post processing needs to be performed on the datamessage or determines that no flow programming needs to be performed forthis data message. At 2020, the process 2000 lets the data messagethrough the egress path of GVM 102, and then ends.

The examples described above by reference to FIGS. 8, 9, 15, 18, and 19show service plane operations that are performed on a data message thatis identified along the egress path of a source GVM. These service planeoperations (described by reference to FIGS. 7, 10-14, 16, 17 and 20) areequally applicable to data messages that are identified as they traversealong the ingress path of a source GVM. To perform these ingress sideoperations, the SI pre and post processors 610 and 612 on the ingresspath are flipped as compared to the locations of these two processors onthe egress path. Specifically, as shown in FIG. 6, the preprocessor 610receives a data message that enters the GVM's ingress path from thesoftware switch port that is associated with this GVM's VNIC, while thepost processor 612 passes the processed data message along the ingressIO chain to the GVM's VNIC.

However, the service insertion and service transport operations for theingress side processing are similar to the egress side processing ofdata messages to and from a particular GVM. In some cases, this GVMexchanges data messages with another GVM. As described above byreference to FIGS. 4 and 5, the service plane can be directed to performthe same service chain on the data messages in each direction, but inthe opposite order. In such cases, the service nodes for the servicepath on the ingress side perform a series of service operations for afirst direction of the service chain for data messages that the otherGVM sends to the particular GVM, while the service nodes for the servicepath on the egress side perform the same series of service operationsbut in a second, opposite direction through the service chain. Also, asmentioned above, the two sets of service nodes for the forward andreverse directions include the same service nodes in some embodiments.

The header formats used in some embodiments will now be described byreference to FIGS. 21, 22, and 25-28. FIG. 21 illustrates an NSH header2100 that some of the service proxies in some embodiments use toencapsulate data messages before providing the data messages to theirassociated service nodes. In some of these embodiments, the servicenodes return the processed data messages encapsulated with such NSHheaders. In some embodiments, the NSH header is also used by the serviceplane modules of host computers to forward double encapsulated datamessages to other host computers, with the first encapsulating headerbeing the NSH header and the second encapsulating header being a servicetransport header. In other embodiments, however, the service insertionand service transport attributes are placed in one encapsulating header,as further described below. Also, as described above and furtherdescribed below, the service proxies and service nodes in someembodiments do not use NSH headers to encapsulate the data messages thatthey exchange.

As shown, all the fields of the first 8 bytes of the NSH header are usedin compliance with RFC 8300. This header includes in some embodiments afixed length metadata (MD) content header 2110. It also includes in someembodiments (1) a MD type, which is set to 1, (2) a next protocol value,which is 3 to signify Ethernet communications, and (3) a length value,which is 6 because the MD content header 2110 has a fixed length. Also,in some embodiments, the SPI and SI fields 2122 and 2124 are filled inwith the service path identifier for the selected path and the currentservice index value, which is the initial SI value (i.e., the initialnumber of service hops) when the pre-processor 610 of the source GVMdefines it.

In some embodiments, the service insertion modules do not store or cachemetadata except in the NSH header carried along with data messages. Inthis model, service nodes preserve the metadata field that they do notintend to change. In some embodiments, certain metadata fields are usedas a communication mechanism for data plane mediated signaling betweenthe service proxies/nodes and the source GVM's service modules. In someembodiments, the data message metadata is encoded over the wire in NSHfixed length context header 2110. In some embodiments, this fixed-sizedheader provides sixteen bytes of information in total. In someembodiments, each service insertion deployment is free to define its ownMD content format.

FIG. 22 illustrates an example of metadata content that is stored in theMD content header 2110 in some embodiments to send service metadata tosuccessive service hops, to service nodes and/or to service proxies. Asshown, this header has sixteen bytes that include a number of fields.One field 2202 contains the F bits, which are used to distinguish thetype of content in the MD content header, e.g., service metadatacontent, flow programming content, etc. In some embodiments, the F bitsfor the service metadata content are b00. Another field 2204 stores a Pbit, which can be set to 1 to force a response to the data message bythe service node. In some embodiments, the response must come with anNSH header containing the same sequence number as that of the requestwith the P bit also set to 1.

The source node identifier (ID) field 2206 unequivocally, for theservice plane, identifies a data compute node (e.g., a GVM) that is thesource or sink of the data message. In some embodiments, the source nodeID includes the service plane MAC address of this source data computenode (DCN) for which the data message was inserted into the serviceplane. The MD content header also includes a sequence number 2208 thatis an opaque 6-bit value that identifies the data message for thepurpose of liveness detection. This value is typically zero unless aservice proxy fills before forwarding the data message to its servicenode as part of its liveness detection.

The MD content header also includes a tenant ID 2212 that identifies atenant uniquely for a multi-tenant datacenter. The tenant ID in someembodiments is a VNI associated with the tenant. The MD content header2200 further includes flow ID 2215 and flow ID validity bit 2222. Insome embodiments, the flow ID validity bit is set to 1 when the rest ofthe flow ID (also called flow tag) is present. The flow ID 2215 is aunique identifier per flow and source DCN (i.e., per flow and sourcenode ID 2206). In some embodiments, the flow ID is set by the sourceDCN's classifier (e.g., the SI pre-processor 610 that performs theclassification operation).

In some embodiments, the flow ID may be discarded when the data messagetraverses a service which is not in native mode (i.e., the service isnot aware of the service plane). In this case, the flow ID is discardedwhen there are not enough bits to carry the flow ID in compatibilitymode headers, which are described below. The flow ID may also bediscarded when a native service (i.e., a service plane aware servicenode) modifies the data message in a way that makes the flow IDmeaningless, for example when a service encrypts traffic from multipleflows into a single IPsec tunnel. In this case preserving the flow tagof the inner data message would be meaningless. In some embodiments, theservice node sets the A bit to zero in this case.

The MD content header 2200 also includes an action field 2230 that isused for flow programming by the service proxies. In some embodiments,the action specifies the action that the source DCN's post-processor 612should perform on a flow. For flow programming, the action fields haveto be non-zero in some embodiments. In addition, for flow programming,the F bits 2202 are also set to 10 and the P bit 2204 is set to 0 at theproxy and ignored by the classifier, and the flow validity bit 2222 andflow tag 2215 have to be valid.

The following are one exemplary set of values for the action field 2230,but one of ordinary skill will realize that other values are specifiedin other embodiments. A value of 0 for the action bit specifies that noflow-programming action is specified. A value of 1 indicates that allmessages of the data message's flow should be dropped at the source, andno further data message of this flow should be forwarded to the serviceplane. Instead, data messages should be dropped at the source afterclassification.

A value of 2 in the action field specifies that the data message shouldbe accepted at the source, and that no further data messages of the sameflow should be forwarded to the service function in some embodiments.Instead, the service function should be skipped and the next service inthe chain invoked directly. A value of 3 in the action field specifiesthat only this data message should be dropped and does not indicate anaction that should be taken on other data messages of the same flow. Insome embodiments, this action is used when the service node communicateswith the service proxy (e.g. to respond to a ping request) and wants adata message to be dropped, even though no flow programming shouldhappen.

The MD content header 2200 also includes a direction field 2214 thatspecifies the direction of the data message from the source DCN tonetwork perspective (e.g., from the DCN to the network is the egressdirection and from the network to the DCN is the ingress direction). Avalue of 0 in the direction field indicates no direction or unknowndirection when the direction is not unknown. In some embodiments, avalue of 1 indicates that the data message is moving in the ingressdirection (i.e., the data message is being processed for source DCN thatis the destination of the data message), for example, the data messageis on its way from a VTEP to its corresponding DCN. A value of 2 in someembodiments indicates an egress direction (e.g., the data message isbeing processed for source DCN that is the source of the data message).

In some embodiments, a value of 3 indicates the data message is merelyin transit and applies to both ingress and egress. When used to define arule, this indicates that the rule should match data messages in onedirection or in any direction in some embodiments. From the serviceperspective, a value of 3 in the direction field indicates that thistraffic was forwarded to the service plane by a transit device that isneither sourcing nor sinking this traffic in some embodiments. In someembodiments, the transit indication is used for traffic that istransiting through a router.

The MD content header 2200 further includes a service chain ID 2216 thatspecifies the service chain along which the data message should flow.Some embodiments do not embed the SCI in the NSH header, and insteadjust store the SPI value. However, other embodiments store the SCI inthe filed 2216 because many SPIs can correspond to the same servicechain and SPIs are also not persistent. In other words, some embodimentsembed the service chain ID because the SCI provides a more stableidentifier for the service nodes to use to identifying service rule thatmatch the data messages that they process.

In some embodiments, other metadata content formats are used internallyby the service plane without being exposed to service nodes, in order toperform data plane signaling between service proxies and servicepost-processor of the source DCN. In some of these embodiments, when theother metadata content formats are used, the OAM bit (the 0 bit 2170 inFIG. 21) of the NSH header is set and no user payload is carried (or, ifany is required by NSH, it is ignored at the destination). In someembodiments, the NSH next protocol field is set to 0 in this case.

In some embodiments, service plane unaware service nodes receive only asubset of the metadata, dependent on the type of non-NSH header used bythe service proxies to communicate with the service nodes. As mentionedabove, the service nodes in some embodiments can receive servicemetadata in GRE headers or in QinQ headers, when the service nodescannot process NSH headers. The GRE and QinQ headers are referred tobelow as compatibility mode headers as they are headers that someexisting service nodes support. Such compatibility mode encapsulationheaders are needed in some embodiments in order to distinguish datamessage flows that are subject to different service processing and toisolate flows with conflicting L3 addresses (in case a single servicenode performs services on data messages of multiple networks, such asmultiple tenant networks).

In some embodiments, a service node in a GRE compatibility mode connectsto its service proxy through two VNICs and is configured inbump-in-the-wire mode. Also, in some embodiments, the VNICs are vmxnet3devices, their MAC addresses do not change, and the MTU size that isused for them is set to a fixed size (e.g., 2048 bytes). One VNIC of theservice node is defined as the unprotected side for receiving egressside traffic and supplying ingress side traffic of the source DCN, whilethe other VNIC is defined as the protected side for receiving ingressside traffic and supplying egress side traffic of the source DCN. Insome embodiments, this information is communicated to a service manageror service node through OVF (Open Virtual Format) parameters, where OVFis a file format that supports exchange of virtual appliances acrossproducts and platforms.

Even though two VNICs are present to support bump-in-the-wire mode, someembodiments use only one service proxy instance per pair ofcompatibility-mode VNICs and use only one endpoint on the service planeto refer to the pair of interfaces. FIGS. 23 and 24 illustrate anexample of a service proxy 2305 forwarding to an SVM 2310 egress-sideand ingress-side data messages of a GVM 2315 with encapsulating GREheaders. To do this, the service proxy creates several virtual GREtunnel endpoints 2320 for the protected and unprotected VNICs of the SVM2310.

Each protected virtual tunnel endpoint has a corresponding unprotectedvirtual tunnel endpoint. Each virtual tunnel endpoint is associated witha virtual IP address, a virtual MAC address and GRE parameters. Theservice proxy encapsulates data messages with GRE headers to traversebetween corresponding pairs of endpoints through the service node, withthis node operating in bump-in-wire mode that does not modify the GREheaders. As further described below, the service proxy embeds servicemetadata in the GRE headers to provide the service node with servicemetadata that it needs to process the data messages. Also, differenttunnel endpoint pairs are used for different flows in some embodiments.

In some embodiments, the service insertion platform supports GREencapsulation as defined in RFC 2784 with the key extension defined inRFC 2890. In some embodiments, GRE tunneling uses IPv4 addresses and theGRE protocol type is set to Transparent Ethernet Bridging as per RFC1701. In the GRE compatibility mode, the service insertion layer (e.g.,the service proxy) generates a tuple (e.g., source IP, destination IP,GRE key) per flow. In some embodiments, this process is deterministicand is based on the contents of the SMD header, which may then bestripped and replaced with the IP and GRE stack. In some embodiments,the IP addresses generated by this process are virtual and are notconfigured on any network entity other than the service proxy and itsassociated SVM, and as a result their scope is limited to the local linkbetween a service proxy and its service node.

The IP address pair and the GRE key are generated in order to carrymetadata along with the data message even when the service node does notsupport GRE. Both the service node and the service proxy in someembodiments consume that metadata. The service node, moreover, isexpected to preserve the outer headers as-is without modifications insome embodiments. In some embodiments, each flow is consistentlyencapsulated in the same GRE tunnel and there can be no IP addressconflicts inside a tunnel. Also, data messages differing only by theirdirection (ingress vs. egress) are encapsulated with the same GRE keywith swapped source and destination IPs and traversing through the GREtunnel endpoints in the proper (protected to unprotected, or unprotectedto protected) direction.

In some embodiments, the IP source/destination addresses, and GRE keycan be inspected by the service node as required to perform the properdata message processing. FIGS. 25 and 26 illustrate how the servicemetadata is encoded in the GRE encapsulation headers in place of thesource and destination IP addresses and GRE key fields. FIG. 25illustrates the GRE header format that is used in some embodiments tostore the service data for the egress direction (e.g., from GVM toswitch), while FIG. 26 illustrates the GRE header format that is used insome embodiments to store the service data for the ingress direction(e.g., from the software switch to the source GVM).

In these figures, all fields are in network byte order. Path IDs aregenerated alongside service paths in some embodiments and have a globalper-service value. As shown in FIGS. 25 and 26, the IP address fieldsare reversed for the egress and ingress side data messages in someembodiments. As with native mode, the service plane in GRE compatibilitymode can modify or generate any traffic as long as it has a validencapsulation when it reaches the service proxy. In some embodiments,this means re-using one of the IP and GRE stacks that the service nodehas received for a related flow.

In some embodiments, the flow tag information along a service chain isdiscarded when entering the first GRE compatibility mode service and isnot restored downstream. This can prevent subsequent services from beingable to declare flow actions. As such, flow programming is not providedto service nodes in GRE compatibility mode of some embodiments. Livenessdetection, moreover, is supported in some embodiments by passing BFD(bidirectional forwarding detection) messages between the trusted anduntrusted interfaces. In some embodiments, these data messages areinjected from the trusted and untrusted sides by the service proxy. Theservice node can recognize this traffic because it is not encapsulatedin GRE. In some embodiments, the service node is expected to forwardthis traffic (and indeed any non-GRE encapsulated traffic) unmodified bybridging it to the other side of the virtual wire. Also, in someembodiments, the data messages can be hard-coded if a real instance ofBFD is not available.

Due to space constrains in some embodiments, certain header fields areencoded in a summarized version. In some embodiments, the service chaintag, SPI and SI are summarized in a single 4-bit field. Eachcompatibility mode service node can therefore be present on at most 16service chain hops in some embodiments. Each time a service is presentinside a service chain, this consumes one service path ID. If theservice is present on multiple chains, multiple service path IDs areconsumed. In addition, each time a service is present on two directionsof a service chain, two service path IDs are consumed.

In some embodiments, locally-generated traffic is supported incompatibility mode as long as a related outer header stack (up to andincluding GRE) is used. In some embodiments, no modification to theouter header stack is allowed, except (1) optionally replacing the outerEthernet destination address with broadcast, (2) updating the IP totalsize field and IP checksum, and (3) the GRE checksum is ignored but theGRE key must be present.

FIGS. 27 and 28 illustrate examples of the encapsulation headers thatare used in some embodiments to send data messages from one VTEPassociated with at least one service node (e.g., from one host computer)to another VTEP associated with another service node (e.g., to anotherhost computer). Both of these examples are Geneve encapsulation headers,and carry the service metadata (e.g., the SMD metadata) in one or moreGeneve TLVs. The Geneve header supports logical L2 overlay transport,and it has a variable TLV space for carrying service-metadata. Hence,different service insertion platforms can specify different amount ofservice metadata to be carried between successive hops.

FIG. 27 illustrate the use of two Geneve encap headers, an outer Geneveheader 2705 for carrying service transport layer data and an innerGeneve header 2710 for carrying service insertion layer metadata. Asshown, the service metadata is stored in an SMD TLV 2715. In someembodiments, this TLV 2715 has the NSH header format of FIG. 21. Hence,this TLV stores the service metadata in the fixed length header 2110 asdescribed above, and stores the SPI and SI values in the SPI and SIfields 2122 and 2124 of the header 2100.

For sake of efficiency, some embodiments combine these two headers intoa single Geneve header 2805 of FIG. 28. To do this, these embodimentsreplace the original source and destination MAC addresses of the datamessage with the service plane MACs of the current and next hops andstore the original source and destination MACs in a new Geneve TLV,along with the service direction, service plane MAC of the source GVM,and other SMD metadata (such as service chain identifier, SPI value, SIvalue, flow programming values, tenant tag, flow tag, etc.). This newGeneve TLV in some embodiments has a 24-byte SMD metadata field, and12-bytes to store STL data, such as the original source and destinationMAC addresses. In some embodiments, the 12-bytes STL data precedes the24-byte SMD metadata, which includes the metadata illustrated in FIGS.21 and 22 in some embodiments.

As shown, in both implementations of FIGS. 27 and 28, the Geneveencapsulating headers store the SVNI of the service plane, which allowsmultiple service planes to be defined. For instance, as described above,some embodiments use the different SVNIs to define different serviceplanes for different entities or tenants in a multi-entity ormulti-tenant datacenter. The different service planes for the differententities or tenants can be associated with the same or different QoSand/or SLA guarantees for the data message types of the entities ortenants. Other embodiments use multiple SVNIs to different serviceplanes for the same entity or tenant, e.g., different service planesassociated with different QoS and/or SLA guarantees for different datamessage types for the same entity or tenant. Also, both headers storethe MAC addresses of the source and destination VTEPs along with the UDPand IP source and destination addresses.

FIG. 29 illustrates an object data model 2900 of some embodiments. Inthis model, objects shown in solid lines are provided by the user, whileobjects shown in dashed lines are generated by the service planemanagers and controllers. As shown, these objects include servicemanagers 2902, services 2904, service profiles 2906, vendor templates2907, a service attachment 2908, service instances 2910, servicedeployment 2913, service instance runtime (SIR) 2912, instance endpoint2914, instance runtime port 2916, service chains 2918, service insertionrules 2920, service paths 2922, and service path hops 2924.

In some embodiments, a service manager object 2902 can be created beforeor after the creation of a service object 2904. An administrator or aservice management system can invoke service manager APIs to create aservice manager. A service manager 2902 can be associated with a serviceat any point of time. In some embodiments, the service manager 2902includes service manager information, such as the vendor name, vendoridentifier, restUrl (for callbacks) and authentication/certificateinformation.

As mentioned above, the service plane does not require the presence oruse of a service manager as service nodes can operate in zero-awarenessmode (i.e., have zero awareness of the service plane). In someembodiments, zero-awareness mode only allows basic operations (e.g.,redirecting traffic towards the service's SVMs). In some suchembodiments, no integration is provided to distribute object information(such as service chain information, service profiles, etc.) to theservice manager servers. Instead, these servers can poll the networkmanager for objects of interest.

A service object 2904 represents a type of service that is provided by aservice node. The service object has a transport type attribute, whichspecifies its mechanism (e.g., NSH, GRE, QinQ, etc.) for receivingservice metadata. Each service object also has a state attribute (whichcan be enabled or disabled) as returned by service manager, and areference to a service manager that may be used for exposing REST APIendpoints to communicate events and perform API calls. It also includesa reference to an OVA/OVF attribute used to deploy instances of theservice.

Vendor template objects 2907 include one or more service profile objects2906. In some embodiments, service managers can register vendortemplates, and the service profiles can be defined on a per servicebasis and based on a vendor template with potentially specializedparameters. A service chain can be defined by reference to one or moreservice profiles. In some embodiments, service profiles are not assignedtags and are not identified explicitly on the wire. In order todetermine which function to apply to traffic, service nodes perform alook up (e.g., based on service chain identifier, service index and theservice direction, as mentioned above) in order to identify theapplicable service profile. The mapping for this lookup is provided bythe management plane to service managers whenever a service chain iscreated of modified.

A service profile object 2906 in some embodiments includes (1) a vendortemplate attribute to identify its associated vendor template, (2) oneor more custom attributes when the template exposes configurable valuesthrough the service profile, and (3) an action attribute, such as aforward action, or a copy-and-redirect, which respectively direct theservice proxies to either forward the received data messages to theirservice nodes, or to forward a copy of the received data messages totheir service nodes while forwarding the received data message to thenext service hop or back to the original source GVM when their servicenode is the last hop.

The service attachment object 2908 represents the service plane (i.e.,is a representation of the service plane of a perspective of a user,such as tenant's network administrator in a multi-tenant datacenter, orthe network administrator in a private datacenter). This serviceattachment object is an abstraction that support any number of differentimplementations of the service plane (e.g., logical L2 overlay, logicalL3 overlay, logical network overlay etc.). In some embodiments, eachendpoint (on an SIR or a GVM) that communicates over the service planespecifies a service attachment. The service attachment is acommunication domain. As such, services or GVMs outside a serviceattachment may not be able to communicate with one another.

In some embodiments, service attachments can be used to create multipleservice planes with hard isolation between them. A service attachmenthas the following attributes (1) logical identifier (e.g., SVNI for alogical switch) that identifies a logical network or logical forwardingelement that carries traffic for the service attachment, (2) a type ofservice attachment (e.g., L2 attachment, L3 attachment, etc.), and (3)an applied_To identifier that specifies a scope of the serviceattachment (e.g., Transport node 0 and Transport node 1 for north-southoperations and a cluster or set of hosts for East-West operations). Insome embodiments, the control plane (e.g., a central control plane)converts the service attachment representation that it receives from themanagement plane to a particular LFE or logical network deployment basedon parameters specified by a network administrator (e.g., a datacenteradministrator of a private or public cloud, or network virtualizationprovider in a public cloud).

A service instance object 2910 represents an actual deployed instancefor a service. Hence, each such object is associated with one serviceobject 2904 through a service deployment object 2913 that specifies therelationship between the service object 2904 and the service instanceobject 2910. The deployed service instance can be a standalone servicenode (e.g., standalone SVM) or it can be a high availability (HA)service node cluster. In some embodiments, the service deployment object2913 describes the service instance type, e.g., standalone or HA. Asdescribed below, the service deployment object's API can be used in someembodiments to deploy several service instances for a service.

The service instance runtime (SIR) object 2912 represents an actualruntime service node that operates in a standalone mode, or an actualruntime service node of an HA cluster. The service instance object insome embodiments includes the following attributes (1) a deployment modeattribute that specifies whether the service instance is operating in astandalone mode, an active/standby mode, or an active/active model, (2)a state attribute that specifies whether the instance is enabled ordisabled, and (3) a deployed to attribute that in the case ofnorth-south operations includes a reference to a service attachmentidentifier.

In some embodiments, SVMs provisioning is initiated manually. To thisend, the management plane provides, in some embodiments, APIs for (1)creating a service instance of an existing service, (2) deleting aservice instance, (3) growing a service instance that is alreadyconfigured as a high availability cluster by adding additional SIRs, and(4) shrinking a service instance by removing one of its SIRs. Whencreating a service instance of an existing service, the service instancemay be created in some embodiments on the basis of a template containedin the service. The caller can pick between a stand-alone instance or anHA cluster, in which case all the VMs in the HA cluster are provisioned.Again, in some embodiments, the API for the service instance deploymentallows multiple service instances (e.g., for an HA cluster) to bedeployed through just one API call.

In some embodiments, an API that creates one or more SVMs specifies oneor more logical locations (e.g. clusters, host, resource pool) in whichthe SVMs should be placed. In some embodiments, the management planetries to place SVMs belonging to the same service instance on differenthosts whenever possible. Anti-affinity rules may also be configured asappropriate to maintain the distribution of SVMs across migration events(such as VMotion events supported by Dynamic Resource Scheduler ofVMware, Inc.). Similarly, the management plane may configure affinityrules with specific hosts (or groups of hosts) when available or theuser provisioning the service instance may explicitly pick a host or acluster.

As mentioned above, a service instance runtime object 2912 represents anactual SVM running on a host to implement a service. An SIR is part of aservice instance. Each SIR can have one or more traffic interfacescompletely dedicated to service plane traffic. In some embodiments, atleast one service proxy instance runs per SIR to handle data planesignaling and data message format conversion for the SIR as needed. Whena service instance is deployed, the SIRs are created for every SVMassociated with the service instance in some embodiments. The networkmanager also creates an instance endpoint for every service instance inan east-west service insertion. Each SIR object 2912 has the followingattributes in some embodiments (1) a state attribute which is active forSVMs that can process traffic and inactive for all others, regardless ofreason, and (2) a runtime state that specifies whether the data planeliveness detection detects that the SIR is up or down.

The instance runtime interface 2916 is the per-endpoint version of theservice instance endpoint 2914. In some embodiments, the instanceruntime interface 2916 is used to identify an interface for an SIR orGVM that can be the source or sink service plane traffic. In East-Westservice insertion, the lifecycle of an instance runtime interface insome embodiments is linked to the lifecycle of the service instanceruntime. In some embodiments, no user action is required to configure aninstance runtime interface.

In some embodiments, the instance runtime interface 2916 has thefollowing attributes: an endpoint identifier, a type, a reference to aservice attachment, and a location. The endpoint identifier is a dataplane identifier for the SIR VNIC. The endpoint identifier is generatedwhen the SIR or GVM is registered with the service transport layer, andmay be a MAC address or part of a MAC address. The type attribute can beshared or dedicated. SIR VNICs are dedicated, meaning that only serviceplane traffic is able to reach them, while GVM VNICs are shared, meaningthey will receive and transmit both service plane and regular traffic.The service-attachment reference is a reference to the serviceattachment that implements the service plane used to transmit andreceive service plane traffic. This reference in some embodiments is tothe SVNI of the service plane. The location attribute in someembodiments specifies the location of the instance runtime interface,which is the UUID of the host on which the instance runtime interface iscurrently located.

In some embodiments, a user defines a service chain object 2918 in termsof an ordered list of service profiles 2906. In some embodiments, eachservice chain conceptually provides separate paths for forward andreverse traffic directions, but if only one direction is provided atcreation time, the other one is generated automatically by reversingservice profile order. Either direction of the service chain (and evenboth directions) can be empty, meaning no services will process trafficin that direction. In some embodiments, the data plane will perform alookup even for an empty service chain.

Service chains are abstract concepts. They do not point to a specificset of service nodes. Rather, the network controllers that are part ofthe service plane platform automatically generate service paths thatpoint to sequences of service nodes for the service chain and directmessages/flows along the generated service paths. In some embodiments, aservice chain is identified in the management plane or control plane byits UUID, a unique identifier of the service chain. Service nodes areprovided with the meaning of service chain IDs through management planeAPIs received through their service managers. One example of this wasdescribed above by reference to FIG. 14.

A service chain tag in some embodiments may be used to identify aservice chain in the dataplane because UUIDs are too long to be carriedin encapsulating headers. A service chain ID in some embodiments is anunsigned integer like rule ID. Each data message redirected to a servicecarries the service chain tag for the service chain it is traversing.The management plane advertises UUID to service chain tag mappings whena service chain is created or modified. Service chain tags have a 1 to 1mapping with service chain UUIDs, whereas a single service chain canhave 0 to many service path indexes.

In addition to a service chain ID, a service chain in some embodimentshas the following attributes: (1) references to all computed servicepaths, (2) failure policies, and (3) references to service profiles.References to computed service paths were described above. The failurepolicy is applied when a service path selected for a service chaincannot be traversed. In some embodiments, the failure policies may bePASS (forward traffic) and FAIL (drop traffic). The references toservice profiles of the service chain may include an egress list ofservice profiles that egress traffic (e.g., data messages traveling froma GVM to a switch) must traverse, and an ingress list of serviceprofiles that ingress traffic (e.g., data messages traveling from theswitch to a GVM) must traverse. In some embodiments, the ingress list isinitialized by default as the reverse of the egress list.

Different techniques can be used in some embodiments to define theservice paths for the service chain. For instance, in some embodiments,a service chain can have an associated load balancing strategy, whichcan be one of the following strategies. The load balancing strategy isresponsible for load balancing traffic across different service paths ofa service chain. According to an ANY strategy, the service framework isfree to redirect the traffic to any service path regardless of any loadbalancing consideration or flow pinning. Another strategy is a LOCALstrategy, which specifies that local service instances (e.g., SVMsexecuting on the same host computer as the source GVM) are to bepreferred over remote service instances (e.g., SVMs executing on otherhost computers or external service appliances).

Some embodiments generate scores for service paths based on how manySIRs are local and the highest score is selected regardless of load.Another strategy is the cluster strategy, which specifies that serviceinstances implemented by VMs that are co-located on the same host arepreferred, whether that host is the local one or a different one. AROUND ROBIN strategy directs that all active service paths are hit withequal probability or based on probabilities that are specified by a setof weight values.

An SI rule object 2920 associates a set of data message attributes witha service chain represented by the service chain object 2918. Theservice chain is implemented by one or more service paths, each of whichis defined by a service path object 2922. Each service path has one ormore service hops, which are represented by one or more service path hopobjects 2924 with each hop being associated with one instance runtimeinterface 2916. Each service hop also refers to an associated serviceprofile, an associated service path, and a next hop SIR endpointidentifier in some embodiments.

In some embodiments, a service path object has several attributes, someof which may be updated by the management or control plane whenunderlying conditions change. These properties include a service pathindex, a state (e.g., enabled or disabled), an administrative mode(e.g., enabled or disabled) used when a service path must be manuallydisabled (e.g., for debugging reasons), a host crossing count(indicating how many times a data message traversing the service pathcrosses hosts), a locality count (indicating how many of the SIRs alongthis path are located on the local host), a list of backup servicepaths, a length of the service path, a reverse path (listing the sameset of SIRs in the reverse order), and a maintenance mode indicator (insome embodiments a bit indicating true if any hop in the service path isin maintenance mode).

The host crossing count is an integer and indicates how many times adata message going through the service path must be sent out of a PNIC.In some embodiments, a local or central control plane uses this metricto determine preferred paths when multiple available alternatives exist.This value is populated by the management plane or control plane and isthe same for each host using the service path. The locality count insome embodiments is not initialized by the management plane or thecontrol plane but rather computed by the local control plane when aservice path is created or updated. Each LCP may potentially compute adifferent number. This value is used by the local control plane toidentify preferred paths when multiple available alternatives exist. Theservice path length is one parameter that is used by the service planeto set the initial service index.

In some embodiments, the list of backup service paths is a pointer to asorted list of all service paths for the same service chain. It listsall possible alternatives to be tried when a specific SIR along the pathis down. This list may contain a service path for all possiblepermutations of SVMs in each HA cluster traversed by the service path.In some embodiments, the list will not contain SIRs belonging todifferent HA clusters.

In some embodiments a service path is disabled when at least one servicehop is inactive. Such a condition is temporary and is triggered byservice liveness detection failures. A service path can be disabled inthis manner at any time. In some embodiments, a service path is alsodisabled when at least one service hop has no matching SIR. The servicehop enters this condition when an SIR it is referring to disappears, butthe service path still exists in the object model.

The service plane must be able to uniquely identify each SPI. In someembodiments, the control plane generated UUIDs are sent for each servicepath. Due to data message header limitations in the service plane, alarge ID is not sent with each data message in some embodiments. In someembodiments, when the control plane generates a UUID for each servicepath, it also generates a small unique ID for it and this ID is sentwith each data message in these embodiments.

FIG. 30 conceptually illustrates several operations that the networkmanagers and controllers perform in some embodiments to define rules forservice insertion, next service hop forwarding, and service processing.As shown, these operations are performed by a service registrator 3004,a service chain creator 3006, a service rule creator 3008, a servicepath generator 3010, a service plane rule generator 3012, and a ruledistributor 3014. In some embodiments, each of these operators can beimplemented by one or more modules of a network manager or controllerand/or can be implemented by one or more standalone servers.

Through a service partner interface 3002 (e.g., a set of APIs or apartner user interface (UI) portal), the service registrator 3004receives vendor templates 3005 that specify services that differentservice partners perform. These templates define the partner services interms of one or more service descriptors, including service profiles.The registrator 3004 stores the service profiles in a profile storage3007 for the service chain creator 3006 to use to define service chains.

Specifically, through a user interface 3018 (e.g., a set of APIs or a UIportal), the service chain creator 3006 receives from a networkadministrator (e.g., a datacenter administrator, a tenant administrator,etc.) one or more service chain definitions. In some embodiments, eachservice chain definition associates a service chain identifier, whichidentified the service chain, with an ordered sequence of one or moreservice profiles. Each service profile in a defined service chain isassociated with a service operation that needs to be performed by aservice node. The service chain creator 3006 stores the definition ofeach service chain in the service chain storage 3020.

Through the user interface 3018 (e.g., a set of APIs or a UI portal),the service rule creator 3008 receives from a network administrator(e.g., a datacenter administrator, a tenant administrator, etc.) one ormore service insertion rules. In some embodiments, each serviceinsertion rule associates a set of data message flow attributes with aservice chain identifier. The flow attributes in some embodiments areflow header attributes, like L2 attributes or L3/L4 attributes (e.g.,five tuple attributes). In these or other embodiments, the flowattributes are contextual attributes (e.g., AppID, process ID, activedirectory ID, etc.). Numerous techniques for capturing and usingcontextual attributes for performing forwarding and service operationsare described in U.S. patent application Ser. No. 15/650,251, which areincorporated herein. Any of these techniques can be used in conjunctionwith the embodiments described herein.

The service rule creator 3008 generates one or more service insertionrules and stores these rules in the SI rule storage 3022. In someembodiments, each service insertion rule has a rule identifier and aservice chain identifier. The rule identifier in some embodiments can bedefined in terms of flow identifiers (e.g., header attributes,contextual attributes, etc.) that identify data message flow(s) to whichthe SI rule is applicable. The service chain identifier of each SI rule,on the other hand, identifies the service chain that has to be performedby the service plane for any data message flow that matches the ruleidentifier of the SI rule.

For each service chain that is part of a service rule, the service pathgenerator 3012 generates one or more service paths, with each pathidentifying one or more service instance endpoints for one or moreservice nodes to perform the service operations specified by the chain'ssequence of service profiles. In some embodiments, the process thatgenerates the service paths for a service chain accounts for one or morecriteria, such as (1) the data message processing load on the servicenodes (e.g., SVMs) that are candidate service nodes for the servicepaths, (2) the number of host computers crossed by the data messages ofa flow as they traverse each candidate service path, etc.

The generation of these service paths is further described in U.S.patent application Ser. No. 16/282,802, which is incorporated herein byreference. As described in this patent application, some embodimentsidentify the service paths to use for a particular GVM on a particularhost based on one or more metrics, such as host crossing count(indicating how many times a data message traversing the service pathcrosses hosts), a locality count (indicating how many of the SIRs alongthis path are located on the local host), etc. Other embodimentsidentify service paths (i.e., select service nodes for service paths)based on other metrics, such as financial and licensing metrics.

The service path generator 3012 stores the identity of the generatedservice paths in the service path storage 3024. This storage in someembodiments associates each service chain identifier to one or moreservice path identifiers, and for each service path (i.e., each SPI) itprovides a list of service instance endpoints that define the servicepath. Some embodiments store the service path definitions in one datastorage, while storing the association between the service chain and itsservice paths in another data storage.

The service rule generator 3010 then generates rules for serviceinsertion, next service hop forwarding, and service processing from therules stored in storages 3020, 3022 and 3024, and stores these rules inrule storages 3026, 3028 and 3030, from where the rule distributor 3014can retrieve these rules and distribute them to the SI pre-processors,service proxies and service nodes. The distributor 3014 also distributesin some embodiments the path definitions from the service path storage3024. The path definitions in some embodiments includes the first hopnetwork address (e.g., MAC address) of the first hop along each path. Insome embodiments, the service rule generator 3010 and/or the ruledistributor 3014 specify and distribute different sets of service pathsfor the same service chain to different host computers, as differentsets of service paths are optimal or preferred for different hostcomputers.

In some embodiments, the SI classification rules that are stored in therule storage 3026 associate flow identifiers with service chainidentifiers. Hence, in some embodiments, the rule generator 3010retrieves these rules form the storage 3022 and stores them in theclassification rule storage 3026. In some embodiments, the ruledistributor 3014 directly retrieves the classification rules from the SIrule storage 3022. For these embodiments, the depiction of the SIclassification rule storage 3026 is more of a conceptual illustration tohighlight the three type of the distributed rules, along with thenext-hop forwarding rules and the service node rules.

In some embodiments, the service rule generator 3010 generates the nexthop forwarding rules for each hop service proxy of each service path foreach service chain. As mentioned above, each service proxy's forwardingtable in some embodiments has a forwarding rule that identifies the nexthop network address for each service path on which the proxy'sassociated service node resides. Each such forwarding rule maps thecurrent SPI/SI values to the next hop network address. The service rulegenerator 3010 generates these rules. For the embodiments in which theSI pre-processor has to look-up the first hop network address, theservice rule generator also generates the first hop look-up rule for theSI pre-processor.

Also, in some embodiments, the service rule generator 3010 generates forthe service nodes service rules that map service chain identifier,service index values and service directions to service profiles of theservice nodes. To do this, the service rule generator uses the servicechain and service path definitions from the storages 3020 and 3024, aswell as the service profile definitions from the service profile storage3007. In some embodiments, the rule distributor forwards the servicenode rules to a service node through a service manager of the servicenode when such a service manager exists. The service profile definitionsare also distributed by the distributor 3014 to the host computers(e.g., to their LCPs) in some embodiments, so that these host computers(e.g., the LCPs) can use these service profiles to configure theirservice proxies, e.g., to configure the service proxies to forwardreceived data messages to their service nodes, or to copy the receiveddata messages and forward the copies to their service nodes, whileforwarding the original received data messages to their next servicenode hops or back to their source GVMs when they are the last hops.

In some embodiments, the management and control plane dynamically modifythe service paths for a service chain, based on the status of theservice nodes of the service paths and the data message processing loadson these service nodes. FIG. 31 illustrates how service paths aredynamically modified in some embodiments. In these embodiments, acentral control plane 3100 works with a local control plane 3110 on thehost computers 3120 to define service paths for a service chain, and tomodify these service paths. The CCP 3100 in some embodiments is acluster of servers (e.g., three servers) that provide control planeoperations for defining configurations based on service rules specifiedby network administrators through a cluster of management servers thatprovide management operations.

As shown, the CCP has a status updater 3102 that receives service nodestatus data from status publishers 3103 on the host computers 3120. Asmentioned above, each time that a service proxy determines that itsassociated service node has failed (e.g., each time a service node failsto respond to the service proxy's liveness signal twice in a row), theservice proxy notifies the LCP 3110 of its host. The LCP then has itsstatus publisher 3103 notify the CCP's status updater 3102 of theservice node's failure.

The status updater 3102 relays any service node failures to the servicepath generator 3012, which in some embodiments is part of the CCP alongwith the SP rule generator 3010 and a statistic collector 3104. Eachtime a service node fails, the service path generator removes from theservice path storage 3024 its previously defined service paths that usethis service node. For each removed service path, the service pathgenerator 3012 deletes or deactivates the removed path's SPI value forthe service chain identifier of the corresponding service chain.

In some embodiments, each removed service path is removed (e.g., deletedor deactivated) from the records of all hosts that previously receivedforwarding rules or path definitions that were for this service path. Insome embodiments, the CCP (e.g., the service path generator 3010 or therule distributor 3014) directs these hosts to remove the service pathfrom the forwarding and path definition rules of their forwarding rulestorages 3128 and path definition storage 808. The LCP of the failedservice node in some embodiments removes the service path from itsforwarding and path definition rules, while in other embodiments eventhis LCP waits for instructions to do so from the CCP.

Each host 3120 also has a statistics publisher 3105 that publishes datamessage load statistics that the service proxies generate for theirservice nodes in some embodiments. Each time a service proxy receives adata message that has been processed by its service node, the serviceproxy in some embodiments increments statistics (e.g., data messagecount, byte count, etc.) that it maintains in a statistic storage 3107for its service node. In some embodiments, the statistics publisher 3105periodically or on-demand retrieves the collected statistics from thestorage 3107 and forwards these statistics to a statistic collector 3104of the CCP. In some embodiments, the statistics collector 3104 receives(through the management plane) statistics that the service managers ofthe service nodes receive from the service nodes.

The statistics collector 3104 relays the collected statistics to theservice path generator 3012. As mentioned above, the service pathgenerator in some embodiments defines the service paths through theservice nodes based in part on the data message load on the servicenodes. For instance, when the data message load on a service nodeexceeds a threshold value, the service path generator performs one ormore actions in some embodiments to reduce the load on this servicenode. For instance, in some embodiments, it stops adding the servicenode to any new service paths that it might define. In these or otherembodiments, it also directs the distributor 3014 to remove the servicepaths that use this service node from some or all of the hosts.

Conjunctively or alternatively, the service path generator directs a CCPmodule (e.g., the distributor 3014) to direct the LCPs of one or morehost computers to adjust the selection criteria 820 used for selectingservice paths that the LCPs generate in order to control how the SIpre-processor performs its path selections. In other embodiments, theservice path generator or another CCP module aggregates the loadstatistics for each service node and distributes the aggregated load tohost LCPs along with their associated SPI values so that the LCPs cananalyze these statistics and adjust the path selection criteria thatthey generate. In some embodiments, each LCP uses or has a pathevaluator 3115 to generate the path selection criteria to evaluate andselect paths based on service node statistics, and/or based on othercriteria, such as number of hosts traversed by each service path.

In some embodiments, the servers that implement the management plane,the control plane, the service managers are in the same datacenter asthe host computers on which the guest and service machines and modules(e.g., GVMs, SVMs, service proxies, port proxies, STL modules, SFEs,etc.) execute. In these embodiments, the management plane servers, thecontrol plane servers, the service managers and the host computermodules (e.g., the LCPs, SVMs, GVMs, hypervisor modules, etc.)communicate with each other through the shared network infrastructure(e.g., the switches, routers, wired and wireless links, etc.) of thedatacenter.

In other embodiments, the management plane servers, the control planeservers, the service managers and/or the host computers operate indifferent datacenters (e.g., enterprise private datacenters and publiccloud datacenters). In some such embodiments, management plane servers,the control plane servers, the service managers and/or the host computermodules (e.g., the LCPs, SVMs, GVMs, hypervisor modules, etc.)communicate with each other through network infrastructures outside oftheir respective datacenters. Also, some such embodiments implement theservice transport layer as a distributed logical L3 routers and/ornetwork that spans multiple datacenters (e.g., multiple privatedatacenters, multiple public datacenters, multiple private/publicdatacenters).

FIG. 32 illustrates a process 3200 that some embodiments perform todefine a service plane and its associated service nodes for a tenant ina multi-tenant datacenter. This process presents just one exemplarysequence of operations and is not meant to convey any required orderingof operations. As shown, the process initially specifies (at 3205) aservice attachment for establishing the service plane. The serviceattachment construct is agnostic to the implementation of the serviceplane. In some embodiments, the service attachment is implemented as alogical switch but, as mentioned above, the service attachment isimplemented differently (e.g., logical router, logical network, etc.) inother embodiments.

Service planes are used in some embodiments to segregate the serviceprocessing for the data traffic of one tenant from the serviceprocessing for the data traffic of other tenants. In these or otherembodiments, different service planes are used to provide different QoSor SLA guarantees for different types of traffic. For example, someembodiments use different service planes to provide different QoS or SLAguarantees for traffic between different data compute endpoints ofdifferent tenants, or different QoS or SLA guarantees for different typeof content carried by different data message flows of the same tenant ordifferent tenants.

After creating the service attachment, the process creates (at 3210)service instances for the services that are to be provided by theservice plane. For each deployed service instance, the process specifieswhether the service instance should be provided by a high availabilitycluster or by a standalone service node. It also provides a serviceattachment identifier that identifies the service attachment associatedwith the service instance. It also provides the deployment specificationand the instance deployment configuration.

Next, at 3215, the process deploys each service instance runtime foreach service instance created at 3210. For each service instanceruntime, an instance endpoint has to be created on the serviceattachment. When the service attachment is a logical switch, the createdinstance endpoint is a logical port of the logical switch. In someembodiments, the logical switch port is auto created when an SVM (thatserves as the service instance runtime) gets attached to the logicalswitch. In some embodiments, the service instance endpoints are createdby the management plane each time a service instance is deployed. Also,in some embodiments, the service instances and service instance runtimesfor a service can be deployed by invoking one service deployment objectAPI. As mentioned above, the use of this single API greatly alleviatesthe need to repeatedly invoke one API multiple times to deploy multipleservice instances and service instance runtimes.

At 3220, the process creates one or more service chains. Each servicechain is created as an ordered list of service profiles. Each servicechain has a forward processing direction and a reverse processingdirection. For each service chain, a failure policy is defined asdescribed above. Also, as described above, the load balancing criteriain some embodiments is defined for each service chain as one of thefollowing types: any, local, service cluster or round robin. Finally, at3225, a section of service rules is defined for the tenant, and one ormore service rules are defined in these sections. Each service rulecorrelates a set of data message flow attributes with a service chainidentifier, in order to specify the service chain that has to beexecuted for data messages that match the specified flow attribute set.

Many of the above-described features and applications are implemented assoftware processes that are specified as a set of instructions recordedon a computer readable storage medium (also referred to as computerreadable medium). When these instructions are executed by one or moreprocessing unit(s) (e.g., one or more processors, cores of processors,or other processing units), they cause the processing unit(s) to performthe actions indicated in the instructions. Examples of computer readablemedia include, but are not limited to, CD-ROMs, flash drives, RAM chips,hard drives, EPROMs, etc. The computer readable media does not includecarrier waves and electronic signals passing wirelessly or over wiredconnections.

In this specification, the term “software” is meant to include firmwareresiding in read-only memory or applications stored in magnetic storage,which can be read into memory for processing by a processor. Also, insome embodiments, multiple software inventions can be implemented assub-parts of a larger program while remaining distinct softwareinventions. In some embodiments, multiple software inventions can alsobe implemented as separate programs. Finally, any combination ofseparate programs that together implement a software invention describedhere is within the scope of the invention. In some embodiments, thesoftware programs, when installed to operate on one or more electronicsystems, define one or more specific machine implementations thatexecute and perform the operations of the software programs.

FIG. 33 conceptually illustrates a computer system 3300 with which someembodiments of the invention are implemented. The computer system 3300can be used to implement any of the above-described hosts, controllers,and managers. As such, it can be used to execute any of the abovedescribed processes. This computer system includes various types ofnon-transitory machine readable media and interfaces for various othertypes of machine readable media. Computer system 3300 includes a bus3305, processing unit(s) 3310, a system memory 3325, a read-only memory3330, a permanent storage device 3335, input devices 3340, and outputdevices 3345.

The bus 3305 collectively represents all system, peripheral, and chipsetbuses that communicatively connect the numerous internal devices of thecomputer system 3300. For instance, the bus 3305 communicativelyconnects the processing unit(s) 3310 with the read-only memory 3330, thesystem memory 3325, and the permanent storage device 3335.

From these various memory units, the processing unit(s) 3310 retrieveinstructions to execute and data to process in order to execute theprocesses of the invention. The processing unit(s) may be a singleprocessor or a multi-core processor in different embodiments. Theread-only-memory (ROM) 3330 stores static data and instructions that areneeded by the processing unit(s) 3310 and other modules of the computersystem. The permanent storage device 3335, on the other hand, is aread-and-write memory device. This device is a non-volatile memory unitthat stores instructions and data even when the computer system 3300 isoff. Some embodiments of the invention use a mass-storage device (suchas a magnetic or optical disk and its corresponding disk drive) as thepermanent storage device 3335.

Other embodiments use a removable storage device (such as a flash drive,etc.) as the permanent storage device. Like the permanent storage device3335, the system memory 3325 is a read-and-write memory device. However,unlike storage device 3335, the system memory is a volatileread-and-write memory, such a random access memory. The system memorystores some of the instructions and data that the processor needs atruntime. In some embodiments, the invention's processes are stored inthe system memory 3325, the permanent storage device 3335, and/or theread-only memory 3330. From these various memory units, the processingunit(s) 3310 retrieve instructions to execute and data to process inorder to execute the processes of some embodiments.

The bus 3305 also connects to the input and output devices 3340 and3345. The input devices enable the user to communicate information andselect commands to the computer system. The input devices 3340 includealphanumeric keyboards and pointing devices (also called “cursor controldevices”). The output devices 3345 display images generated by thecomputer system. The output devices include printers and displaydevices, such as cathode ray tubes (CRT) or liquid crystal displays(LCD). Some embodiments include devices such as a touchscreen thatfunction as both input and output devices.

Finally, as shown in FIG. 33, bus 3305 also couples computer system 3300to a network 3365 through a network adapter (not shown). In this manner,the computer can be a part of a network of computers (such as a localarea network (“LAN”), a wide area network (“WAN”), or an Intranet, or anetwork of networks, such as the Internet. Any or all components ofcomputer system 3300 may be used in conjunction with the invention.

Some embodiments include electronic components, such as microprocessors,storage and memory that store computer program instructions in amachine-readable or computer-readable medium (alternatively referred toas computer-readable storage media, machine-readable media, ormachine-readable storage media). Some examples of such computer-readablemedia include RAM, ROM, read-only compact discs (CD-ROM), recordablecompact discs (CD-R), rewritable compact discs (CD-RW), read-onlydigital versatile discs (e.g., DVD-ROM, dual-layer DVD-ROM), a varietyof recordable/rewritable DVDs (e.g., DVD-RAM, DVD-RW, DVD+RW, etc.),flash memory (e.g., SD cards, mini-SD cards, micro-SD cards, etc.),magnetic and/or solid state hard drives, read-only and recordableBlu-Ray® discs, ultra-density optical discs, and any other optical ormagnetic media. The computer-readable media may store a computer programthat is executable by at least one processing unit and includes sets ofinstructions for performing various operations. Examples of computerprograms or computer code include machine code, such as is produced by acompiler, and files including higher-level code that are executed by acomputer, an electronic component, or a microprocessor using aninterpreter.

While the above discussion primarily refers to microprocessor ormulti-core processors that execute software, some embodiments areperformed by one or more integrated circuits, such as applicationspecific integrated circuits (ASICs) or field programmable gate arrays(FPGAs). In some embodiments, such integrated circuits executeinstructions that are stored on the circuit itself.

As used in this specification, the terms “computer”, “server”,“processor”, and “memory” all refer to electronic or other technologicaldevices. These terms exclude people or groups of people. For thepurposes of the specification, the terms display or displaying meansdisplaying on an electronic device. As used in this specification, theterms “computer readable medium,” “computer readable media,” and“machine readable medium” are entirely restricted to tangible, physicalobjects that store information in a form that is readable by a computer.These terms exclude any wireless signals, wired download signals, andany other ephemeral or transitory signals.

While the invention has been described with reference to numerousspecific details, one of ordinary skill in the art will recognize thatthe invention can be embodied in other specific forms without departingfrom the spirit of the invention. For instance, several figuresconceptually illustrate processes. The specific operations of theseprocesses may not be performed in the exact order shown and described.The specific operations may not be performed in one continuous series ofoperations, and different specific operations may be performed indifferent embodiments. Furthermore, the process could be implementedusing several sub-processes, or as part of a larger macro process.

Even though the service insertion rules in several of theabove-described examples provide service chain identifiers, some of theinventions described herein can be implemented by having a serviceinsertion rule provide the service identifiers (e.g., SPIs) of thedifferent services specified by the service insertion rule. Similarly,several of the above-described embodiments perform distributed servicerouting that relies at each service hop identifying a next service hopby performing an exact match based on the SPI/SI values. However, someof the inventions described herein can be implemented by having theservice insertion pre-processor embed all the service hop identifiers(e.g., service hop MAC addresses) as the data message's serviceattribute set and/or in the data message's encapsulating service header.

In addition, some embodiments decrement the SI value differently (e.g.,at different times) than the approaches described above. Also, insteadof performing the next hop lookup just based on the SPI and SI values,some embodiments perform this lookup based on the SPI, SI and servicedirection values as these embodiments use a common SPI value for boththe forward and reverse directions of data messages flowing between twomachines.

The above-described methodology is used in some embodiments to expresspath information in single tenant environments. Thus, one of ordinaryskill will realize that some embodiments of the invention are equallyapplicable to single tenant datacenters. Conversely, in someembodiments, the above-described methodology is used to carry pathinformation across different datacenters of different datacenterproviders when one entity (e.g., one corporation) is a tenant inmultiple different datacenters of different providers. In theseembodiments, the tenant identifiers that are embedded in the tunnelheaders have to be unique across the datacenters, or have to betranslated when they traverse from one datacenter to the next. Thus, oneof ordinary skill in the art would understand that the invention is notto be limited by the foregoing illustrative details, but rather is to bedefined by the appended claims

1. A method of performing a sequence of services on data message flowson a host computer, the method comprising: at the host computer:receiving data regarding a plurality of service paths each of whichspecifies a set of service nodes for performing the sequence ofservices; receiving load balancing criteria to use to select differentservice paths for different data message flows when each of theplurality of service paths is applicable to the data message flows;receiving different data messages for different data message flowsassociated with a machine executing on a host computer; using the loadbalancing criteria to select different service paths for the differentdata message flows; forwarding different data message flows along thedifferent service paths to perform the same set of services on the datamessages of the different flows.
 2. The method of claim 1, wherein theload balancing criteria includes round robin criterion that distributesthe load for processing different data message flows with differentservice paths in the plurality of service paths.
 3. The method of claim2, wherein the round robin criterion comprises a plurality of weightvalues associated with the service paths.
 4. The method of claim 1,wherein the load balancing criteria includes a plurality of numericalranges with different ranges associated with different service paths,using the loading balancing criteria comprises generating numericalvalues for each data message flow and mapping each flow's generatednumerical value to one of the numerical ranges in order to select theservice path associated with numerical range as the service path of theflow.
 5. The method of claim 4, wherein the generating the numericalvalues comprises using a random number generator to generate a random orpseudo random number for a data message flow from a set of attributesassociated with the data message flow.
 6. The method of claim 4, whereinthe generating the numerical values comprises using a deterministicfunction to generate a determined numerical value for each data messageflow from a set of attributes associated with the data message flow. 7.The method of claim 1 further comprising: receiving a service insertionrule that comprises a rule identifier and specifies a service chainidentifier for data message flows that match the rule identifier, theservice chain identifier specifying a service chain that is defined toinclude the sequence of services; receiving a record that maps theservice chain identifier to a plurality of service path identifiers;determining that a particular data message of a particular data messageflow matches the rule identifier of a particular service insertion rule;using the service chain identifier of the matched service insertion ruleto identify the received set of service paths.
 8. The method of claim 1further comprising receiving, at the host computer, updates to the loadbalancing criteria to adjust the selection of the different servicepaths to the different data message flows.
 9. The method of claim 8,wherein the updated load balancing criteria is generated based onstatistics regarding data message processing by the service nodes. 10.The method of claim 9 further comprising: at the host computer,collecting statistics regarding data message processing by at least oneservice node executing on the host computer; and providing the collectedstatistics to a server in order for the statistics to be used to updatethe load balancing criteria.
 11. A non-transitory machine readablemedium storing a program for execution by at least one processing unitof a host computer and for performing a sequence of services on datamessage flows on the host computer, the program comprising sets ofinstructions for: receiving data regarding a plurality of service pathseach of which specifies a set of service nodes for performing thesequence of services; receiving load balancing criteria to use to selectdifferent service paths for different data message flows when each ofthe plurality of service paths is applicable to the data message flows;receiving different data messages for different data message flowsassociated with a machine executing on a host computer; using the loadbalancing criteria to select different service paths for the differentdata message flows; forwarding different data message flows along thedifferent service paths to perform the same set of services on the datamessages of the different flows.
 12. The non-transitory machine readablemedium of claim 11, wherein the load balancing criteria includes roundrobin criterion that distributes the load for processing different datamessage flows with different service paths in the plurality of servicepaths.
 13. The non-transitory machine readable medium of claim 12,wherein the round robin criterion comprises a plurality of weight valuesassociated with the service paths.
 14. The non-transitory machinereadable medium of claim 11, wherein the load balancing criteriaincludes a plurality of numerical ranges with different rangesassociated with different service paths, the set of instructions forusing the loading balancing criteria comprises a set of instructions forgenerating numerical values for each data message flow and mapping eachflow's generated numerical value to one of the numerical ranges in orderto select the service path associated with numerical range as theservice path of the flow.
 15. The non-transitory machine readable mediumof claim 14, wherein the set of instructions for generating thenumerical values comprises a set of instructions for using a randomnumber generator to generate a random or pseudo random number for a datamessage flow from a set of attributes associated with the data messageflow.
 16. The non-transitory machine readable medium of claim 14,wherein the set of instructions for generating the numerical valuescomprises a set of instructions for using a deterministic function togenerate a determined numerical value for each data message flow from aset of attributes associated with the data message flow.
 17. Thenon-transitory machine readable medium of claim 11, wherein the programfurther comprises sets of instructions for: receiving a serviceinsertion rule that comprises a rule identifier and specifies a servicechain identifier for data message flows that match the rule identifier,the service chain identifier specifying a service chain that is definedto include the sequence of services; receiving a record that maps theservice chain identifier to a plurality of service path identifiers;determining that a particular data message of a particular data messageflow matches the rule identifier of a particular service insertion rule;using the service chain identifier of the matched service insertion ruleto identify the received set of service paths.
 18. The non-transitorymachine readable medium of claim 11, wherein the program furthercomprises a set of instructions for receiving, at the host computer,updates to the load balancing criteria to adjust the selection of thedifferent service paths to the different data message flows.
 19. Thenon-transitory machine readable medium of claim 18, wherein the updatedload balancing criteria is generated based on statistics regarding datamessage processing by the service nodes.
 20. The non-transitory machinereadable medium of claim 19, wherein the program further comprises setsof instructions for: collecting statistics regarding data messageprocessing by at least one service node executing on the host computer;and providing the collected statistics to a server in order for thestatistics to be used to update the load balancing criteria.